-- Logs begin at Wed 2022-09-28 11:41:10 UTC, end at Wed 2022-09-28 12:48:15 UTC. -- Sep 28 12:08:01.063021 dv-cinder-844563-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 28 12:08:01.070364 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 28 12:08:01.072367 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 28 12:08:01 2022] *** Sep 28 12:08:01.072367 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 28 12:08:01.072367 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 28 12:08:01.072367 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: nodename: dv-cinder-844563-9-iscsi Sep 28 12:08:01.072367 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: machine: x86_64 Sep 28 12:08:01.072367 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: clock source: unix Sep 28 12:08:01.072367 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: pcre jit disabled Sep 28 12:08:01.072367 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: detected number of CPU cores: 4 Sep 28 12:08:01.072863 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: current working directory: / Sep 28 12:08:01.072863 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: detected binary path: /usr/bin/uwsgi-core Sep 28 12:08:01.072863 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: your processes number limit is 47680 Sep 28 12:08:01.072863 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: your memory page size is 4096 bytes Sep 28 12:08:01.072863 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: detected max file descriptor number: 2048 Sep 28 12:08:01.072863 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: lock engine: pthread robust mutexes Sep 28 12:08:01.072863 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: thunder lock: enabled Sep 28 12:08:01.072863 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 28 12:08:01.072863 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 28 12:08:01.113364 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: Python main interpreter initialized at 0x55f94d3829c0 Sep 28 12:08:01.113364 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: python threads support enabled Sep 28 12:08:01.113364 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: your server socket listen backlog is limited to 100 connections Sep 28 12:08:01.113364 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: your mercy for graceful operations on workers is 90 seconds Sep 28 12:08:01.113839 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: mapped 403077 bytes (393 KB) for 2 cores Sep 28 12:08:01.113956 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: *** Operational MODE: preforking *** Sep 28 12:08:01.114255 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: *** uWSGI is running in multiple interpreter mode *** Sep 28 12:08:01.114255 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: spawned uWSGI master process (pid: 81370) Sep 28 12:08:01.114458 dv-cinder-844563-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 28 12:08:01.116138 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: spawned uWSGI worker 1 (pid: 81384, cores: 1) Sep 28 12:08:01.116405 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: spawned uWSGI worker 2 (pid: 81385, cores: 1) Sep 28 12:08:01.116564 dv-cinder-844563-9-iscsi devstack@keystone.service[81370]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 28 12:08:05.419803 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 12:08:05.419803 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: from cryptography.utils import int_from_bytes Sep 28 12:08:05.420641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 12:08:05.420641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: from cryptography.utils import int_from_bytes Sep 28 12:08:05.423847 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 12:08:05.423847 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: from cryptography.utils import int_from_bytes Sep 28 12:08:05.424316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 12:08:05.424316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: from cryptography.utils import int_from_bytes Sep 28 12:08:05.470740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 28 12:08:05.470940 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 28 12:08:05.471138 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] command line args: [] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 28 12:08:05.471412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 28 12:08:05.471594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ================================================================================ {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 28 12:08:05.471679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 28 12:08:05.471852 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] admin_token = **** {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.471937 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 28 12:08:05.472057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] command line args: [] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 28 12:08:05.472141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] config_dir = [] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.472279 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.472355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 28 12:08:05.472465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] config_source = [] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.472543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ================================================================================ {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 28 12:08:05.472649 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] debug = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.473281 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] admin_token = **** {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.473420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.474791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] config_dir = [] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.474924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.475003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.475112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.475189 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] config_source = [] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.475265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.475390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] debug = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.475471 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.475549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.475642 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] list_limit = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.475713 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.475792 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] log_config_append = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.475862 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.475976 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.476048 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.476124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] log_dir = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.476124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] log_file = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.476297 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.476376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.476489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] list_limit = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.476489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] log_config_append = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.476672 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.476746 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.476866 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.476940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] log_dir = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.476940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] log_file = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.477122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.477122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.477339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.477416 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.477557 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.477675 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.477794 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.477868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.477993 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.478074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.478145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.478145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.478317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.478443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.478526 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.478526 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.478709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.478709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.478890 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.478969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.479085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.479165 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.479250 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.479321 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.479321 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.479452 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.479533 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.479599 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] publish_errors = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.479667 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.479740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.479846 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.480029 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.480103 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.480265 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] publish_errors = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.480352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.480501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.480583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.480733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.480826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] standard_threads = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.480904 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.480981 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.481165 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.481258 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.481400 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.481484 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.481582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] use_journal = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.481665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] standard_threads = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.481763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] use_json = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.481845 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.481941 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] use_stderr = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.482069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.482149 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] use_syslog = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.482248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.482345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.482426 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] use_journal = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.482594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] use_json = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.482682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.482755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] use_stderr = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.482823 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.482933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] use_syslog = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.483088 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.483171 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 12:08:05.483331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.483409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.483588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.483666 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.483834 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.483916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.484089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.484167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.484330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.484413 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.484566 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.484647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.484815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.484893 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.485054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.485144 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.485332 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.485408 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.485582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.485659 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.485820 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.485895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.486053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.486393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.486569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.486654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.486808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.486892 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.487058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.487135 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.487308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.487392 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.487548 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.487625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.487787 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.487940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.488048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.488125 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.488292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.488367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.489196 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.489345 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.489498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.489665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.489800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.489927 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.490054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.490203 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.490329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.490454 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.490579 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.490709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.490834 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.490966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.491098 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.491222 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.491352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.491483 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.491614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.491748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.491889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.492013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.492140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.492264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.492393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.492524 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.492649 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.492773 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.492897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.493027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.493174 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.493311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.493445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.493570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.493707 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.493838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.493964 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.494097 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.494225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.494356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.494481 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.494606 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.494731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.494936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.495053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.495175 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.495303 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.495436 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.495571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.495704 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.495826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.495946 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.496067 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.496187 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.496308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.496439 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.496561 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.496682 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.496803 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.496923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.497044 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.497434 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.497514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.497581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.497641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.497699 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.497761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.497828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.497888 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.497946 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.498035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.498102 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.498169 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.498429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.498563 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.498688 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.498807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.498936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.499065 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.499186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.499186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.499425 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.499557 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.499683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.499814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.499949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.499949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.500194 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.500194 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.500426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.500547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.500667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.500795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.500917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.501037 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.501186 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.501307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.501441 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.501571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.501693 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.501813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.501944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.502071 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.502191 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.502319 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.502441 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.502562 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.502681 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.502805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.502927 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.503059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.503182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.503317 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.503449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.503575 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.503710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.503847 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.503972 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.504108 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.504259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.504432 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.504563 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.504699 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.504823 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.504949 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.505076 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.505275 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.505400 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.505547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.505675 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.505801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.505923 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.506047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.506173 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.506308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.506432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.506561 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.506691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.506823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.506947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.507147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.507274 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.507411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.507535 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.507658 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.507791 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.507923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.508053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.508179 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.508309 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.508434 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.508558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.508682 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.508816 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.508949 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.509073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.509231 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.509363 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.509489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.509622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.509747 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.509871 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.509996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.510119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.510262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.510262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.510505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.510505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.510743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.510879 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] auth.external = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.511028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.511163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.511288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.511288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] auth.external = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.511537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.511664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.511790 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.511790 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] auth.password = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.512026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.512160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] auth.token = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.512286 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.512286 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] auth.password = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.512521 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.512655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] auth.token = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.512779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.512912 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.513038 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.513227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.513360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.513487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.513614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.513745 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.513870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.513994 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.514127 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.514251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.514385 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] credential.caching = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.514518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.514643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.514774 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] credential.caching = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.514898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.515023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.515149 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.515285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.515413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.515537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.515662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.515787 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.515912 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.516043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.516181 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.516308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.516433 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.516557 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.516682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.516818 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.516944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.517068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.517214 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.517338 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.517464 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.517602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.517677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.517739 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.517807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.517869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.517928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.518086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.518206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] federation.caching = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.518330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.518452 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.518579 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] federation.caching = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.518701 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.518830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.518953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.519075 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.519197 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.519336 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.519467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.519592 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.519714 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.519833 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.519956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.520077 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.520206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.520329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.520451 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.520574 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.520694 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.520899 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.521040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.521194 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.521318 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] identity.caching = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.521445 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.521567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.521693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] identity.caching = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.521894 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.522022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.522145 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.522270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.522393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.522520 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.522654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.522780 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.522914 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.523039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.523163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.523163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.523404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.523404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.523637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.523637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.523873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.523998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.524129 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.524254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.524377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.524501 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.524636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.524636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.524880 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.525005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.525147 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.525281 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.525404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.525526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.525656 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.525780 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.525902 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.526024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.526145 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.526277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.526408 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.526531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.526653 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.526776 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.526897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.527019 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.527148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.527271 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.527393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.527515 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.527636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.527758 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.527889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.528022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.528145 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.528267 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.528388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.528510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.528639 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.528764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.528886 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.529008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.529169 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.529309 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.529436 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.529562 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.529697 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.529824 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.529947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.530078 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.530203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.530327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.530450 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.530573 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.530696 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.530829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.530956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.531080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.531202 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.531328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.531461 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.531598 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.531725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.531849 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.531975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.532099 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.532222 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.532354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.532479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.532603 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.532726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.532850 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.532975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.533107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.533107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.533370 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.533370 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.533609 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.533745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.533879 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.533879 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.534118 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.534118 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.534356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.534482 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.534616 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.534743 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.534879 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.535006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.535132 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.535763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.535895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.536019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.536141 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.536222 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.536332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.536459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.536588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.536710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.536832 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.536952 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.537085 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.537265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.537399 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.537530 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.537655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.537784 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.537910 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.538032 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.538166 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.538294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.538417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.538540 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.538666 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.538804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.538938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.539063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.539191 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.539317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.539441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.539564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.539696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.539820 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.539945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.540068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.540192 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.540317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.540450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.540585 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.540710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.540839 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.540964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.541087 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.541247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.541376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.541502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.541628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.541755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.541880 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.542014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.542141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.542266 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.542402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.542529 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.542654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.542789 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.542914 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.543040 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.543166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.543292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.543418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.543553 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.543680 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.543806 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.543931 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.544056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.544192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] resource.caching = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.544327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.544453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.544580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] resource.caching = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.544654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.544715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.544773 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.544841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.545020 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.545169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.545292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.545417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.545579 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.545717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.545844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.545967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.546092 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.546217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.546417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.546544 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.546671 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] role.caching = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.546796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.546920 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] role.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.547043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] role.caching = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.547173 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.547298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] role.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.547421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.547556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.547683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.547807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.547940 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.548065 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.548190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.548314 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.548444 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.548569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.548702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.548827 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.548951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.549033 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.549182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.549313 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.549445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.549574 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.549701 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.549823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.549943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.550065 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.550199 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.550328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.550449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.550575 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.550575 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.550814 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.550814 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.551185 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.551314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.551558 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.551683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.551806 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.551937 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.552066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.552193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.552321 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.552443 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.552572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.552710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.552840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.552968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.553095 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.553259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.553388 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.553514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.553638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.553762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.553886 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.554013 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.554145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.554270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.554395 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.554529 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.554654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.554778 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.554909 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.555036 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.555169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.555291 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.555414 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.555539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.555670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.555752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] token.caching = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.555907 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.556033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.556168 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] token.caching = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.556168 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.556413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.556413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.556648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.556648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.556913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.557040 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.557196 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.557315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.557438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.557560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.557681 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.557803 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.557945 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.558069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.558193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.558315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.558436 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.558559 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.558688 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.558811 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.558933 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.559055 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.559177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.559299 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.559437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.559560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.559692 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.559823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.559964 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.560099 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.560223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.560346 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.560469 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.connection = **** {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.560599 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.connection = **** {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.560722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.560844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.561010 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.561180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.561307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.561444 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.561586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.561713 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.561840 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.561966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.562092 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.562225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.562352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.562476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.562601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.562724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.562848 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.562980 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.563106 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.563242 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.563369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.563494 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.563619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.563752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.563879 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.564006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.564130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.564255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.564379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.564512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.564638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.564763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.564887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.565020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.565163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.565297 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 12:08:05.565425 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81385) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 28 12:08:05.565554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81384) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 28 12:08:05.589025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.589284 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.589384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.589513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.589596 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.589799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.598457 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.598861 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.626590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.626820 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.627156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.627395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.680600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.681093 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.729771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.730006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.730092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.730245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.886431 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.886619 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.886729 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.886807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.886881 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.886978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.887094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.887177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.887320 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.887403 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.887536 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.887619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.911328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.911432 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.911564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.911647 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.911796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.911879 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.912025 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.912107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.912250 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.912342 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.912467 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.912549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.912685 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.912766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.923819 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.923971 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.936649 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.936761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.936840 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.936926 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.937061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.937179 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.937321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.937403 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.937550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.937646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.937774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.937857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.937998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81385) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.938080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81384) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 12:08:05.955318 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.955532 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 12:08:05.955642 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.955854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 12:08:05.955934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.956069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 12:08:05.956202 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.956363 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.956453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 12:08:05.956623 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.956706 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.956894 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.956977 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.957241 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.957325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.957529 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.966885 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.967093 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.967226 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.967375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.967504 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.967654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.967803 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.967933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.975387 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:05.975526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:05.978355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:05.978503 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:05.978631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.978761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.978918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.979046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.979187 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.979325 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.979476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.979604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.979748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.979871 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.980021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.980149 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.980274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.980397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.980521 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.980663 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.980791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.980915 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.981038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:05.981190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.004521 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.004932 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.006300 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.006442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.006593 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.006721 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.011258 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.011434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.011563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.011726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.011854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.012016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.018149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.018437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.018568 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.018734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.024201 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.024500 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.024630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.024791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.024920 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.025081 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.025233 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.025387 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.025515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.025667 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.025796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.025938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.026070 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.026196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.026320 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.026483 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.042202 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.042485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.042615 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.042768 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.046300 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.046635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.049518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.049816 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.049949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.050151 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.054084 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.054484 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.057154 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.057587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.060328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.060634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.060783 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.060911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.061063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.061216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.061387 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.061546 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.061672 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.061796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.061955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.062245 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.078645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.078949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.079229 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.079503 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.081259 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.081773 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.082887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.083359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.087547 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.089125 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.090709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.090996 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.091415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.092961 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.093827 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.094130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.094425 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.094562 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.094719 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.094849 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.095133 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.097557 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.097876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.098167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.098559 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.098972 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.106742 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.107040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.107334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.107627 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.108022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.108311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.108607 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.110108 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.110402 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.110690 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.110978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.111630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.111926 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.112210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.124431 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.127398 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.128607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.130965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.132421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.132905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.134051 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.134344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.134756 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.136314 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.137841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.139329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.139619 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.140028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.141172 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.142655 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.145661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.146063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.146360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.146651 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.146787 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.148127 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.151054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.151458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.151750 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.152030 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.154493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.154792 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.155081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.159839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.160137 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.161070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.162597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81385) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.162889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.163182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.163479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.163762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.164051 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.164335 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.164616 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.164894 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.165220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.165525 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.165878 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.166076 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.166370 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81385) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.167968 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81384) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 12:08:06.168255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.168535 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.168814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.169091 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.169397 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.169675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.170044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.170223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.170501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.170790 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.171055 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.171322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.171604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81384) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 12:08:06.202219 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f94d3829c0 pid: 81385 (default app) Sep 28 12:08:06.204620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b275f-9992-4022-8bc1-6ee5e3dc2af8 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:06.204916 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b275f-9992-4022-8bc1-6ee5e3dc2af8 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:06.205218 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b275f-9992-4022-8bc1-6ee5e3dc2af8 None None] PATH_INFO: `/v3` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:06.206956 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f94d3829c0 pid: 81384 (default app) Sep 28 12:08:06.209560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1/1] 10.222.0.60 () {50 vars in 791 bytes} [Wed Sep 28 12:08:06 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:12.983422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657038b-9fb1-4a15-b264-938bb4c8d6a6 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:12.983942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657038b-9fb1-4a15-b264-938bb4c8d6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:12.984048 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657038b-9fb1-4a15-b264-938bb4c8d6a6 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:12.988582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1/2] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:12.994065 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0307a6-70ca-4f94-b3b9-a5b6302c4bb2 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:12.994608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0307a6-70ca-4f94-b3b9-a5b6302c4bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:12.994608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0307a6-70ca-4f94-b3b9-a5b6302c4bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:13.067595 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: INFO dbcounter [None req-0c0307a6-70ca-4f94-b3b9-a5b6302c4bb2 None None] Registered counter for database keystone Sep 28 12:08:13.086101 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG oslo_db.sqlalchemy.engines [None req-0c0307a6-70ca-4f94-b3b9-a5b6302c4bb2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81385) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 28 12:08:13.092241 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writer thread running {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 28 12:08:13.273896 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG passlib.handlers.bcrypt [None req-0c0307a6-70ca-4f94-b3b9-a5b6302c4bb2 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81385) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 28 12:08:13.274198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG passlib.handlers.bcrypt [None req-0c0307a6-70ca-4f94-b3b9-a5b6302c4bb2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81385) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 28 12:08:13.332071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-0c0307a6-70ca-4f94-b3b9-a5b6302c4bb2 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:13.333957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0c0307a6-70ca-4f94-b3b9-a5b6302c4bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:13.460892 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 2/3] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 28 12:08:14.217322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657038b-9fb1-4a15-b264-938bb4c8d6a6 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:14.286915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: INFO dbcounter [None req-6657038b-9fb1-4a15-b264-938bb4c8d6a6 None None] Registered counter for database keystone Sep 28 12:08:14.307648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG oslo_db.sqlalchemy.engines [None req-6657038b-9fb1-4a15-b264-938bb4c8d6a6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81384) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 28 12:08:14.313841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writer thread running {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 28 12:08:14.430651 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657038b-9fb1-4a15-b264-938bb4c8d6a6 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:14.434861 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e21d2-cec0-4ee9-b6f8-19868943e51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:14.435477 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e21d2-cec0-4ee9-b6f8-19868943e51c admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:14.435753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e21d2-cec0-4ee9-b6f8-19868943e51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:14.436020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e21d2-cec0-4ee9-b6f8-19868943e51c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:14.444568 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e21d2-cec0-4ee9-b6f8-19868943e51c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:14.496219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ba7e21d2-cec0-4ee9-b6f8-19868943e51c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 28 12:08:14.496557 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e21d2-cec0-4ee9-b6f8-19868943e51c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:14.506249 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 2/4] 10.222.0.60 () {58 vars in 1324 bytes} [Wed Sep 28 12:08:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 12:08:14.512122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0307a6-70ca-4f94-b3b9-a5b6302c4bb2 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:14.529107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0307a6-70ca-4f94-b3b9-a5b6302c4bb2 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:14.533508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e355837-0f2e-4d4c-9a1d-c283bb8c835d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:14.534869 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e355837-0f2e-4d4c-9a1d-c283bb8c835d admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:14.535195 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e355837-0f2e-4d4c-9a1d-c283bb8c835d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:14.535468 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e355837-0f2e-4d4c-9a1d-c283bb8c835d admin admin] PATH_INFO: `/v3/projects/2fc00be567cf4966a588e131d48e5a69` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:14.537050 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e355837-0f2e-4d4c-9a1d-c283bb8c835d admin admin] RBAC: Authorizing `identity:get_project(project_id=2fc00be567cf4966a588e131d48e5a69)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:14.590003 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5e355837-0f2e-4d4c-9a1d-c283bb8c835d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 28 12:08:14.590349 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e355837-0f2e-4d4c-9a1d-c283bb8c835d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:14.594786 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 3/5] 10.222.0.60 () {56 vars in 1207 bytes} [Wed Sep 28 12:08:14 2022] GET /identity/v3/projects/2fc00be567cf4966a588e131d48e5a69 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:16.700487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85239781-de7c-402c-bbe8-a159ec5c533b None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:16.701001 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85239781-de7c-402c-bbe8-a159ec5c533b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:16.701001 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85239781-de7c-402c-bbe8-a159ec5c533b None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:16.705090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 3/6] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:16.711036 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee138e8-b481-4039-b02a-e945c5181054 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:16.711436 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee138e8-b481-4039-b02a-e945c5181054 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:16.711580 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee138e8-b481-4039-b02a-e945c5181054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:16.731122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-bee138e8-b481-4039-b02a-e945c5181054 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:16.732625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-bee138e8-b481-4039-b02a-e945c5181054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:16.740768 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 4/7] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 28 12:08:17.010361 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85239781-de7c-402c-bbe8-a159ec5c533b None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:17.022751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85239781-de7c-402c-bbe8-a159ec5c533b None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:17.026839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0aa4d-0779-4d1f-ba44-b86c55f7680b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:17.027423 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0aa4d-0779-4d1f-ba44-b86c55f7680b admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:17.027702 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0aa4d-0779-4d1f-ba44-b86c55f7680b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:17.027971 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0aa4d-0779-4d1f-ba44-b86c55f7680b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:17.036352 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0aa4d-0779-4d1f-ba44-b86c55f7680b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:17.038127 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0aa4d-0779-4d1f-ba44-b86c55f7680b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:17.046564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 4/8] 10.222.0.60 () {58 vars in 1324 bytes} [Wed Sep 28 12:08:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 12:08:17.052081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee138e8-b481-4039-b02a-e945c5181054 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:17.061157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee138e8-b481-4039-b02a-e945c5181054 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:17.065493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397de1f9-a58a-4f3b-bd83-f21d6db92ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:17.066130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397de1f9-a58a-4f3b-bd83-f21d6db92ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:17.066508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397de1f9-a58a-4f3b-bd83-f21d6db92ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:17.066787 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397de1f9-a58a-4f3b-bd83-f21d6db92ea9 admin admin] PATH_INFO: `/v3/users/1290a1aa40a149388a72943a7b2dc68e` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:17.068090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397de1f9-a58a-4f3b-bd83-f21d6db92ea9 admin admin] RBAC: Authorizing `identity:get_user(user_id=1290a1aa40a149388a72943a7b2dc68e)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:17.069964 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397de1f9-a58a-4f3b-bd83-f21d6db92ea9 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:17.071474 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 5/9] 10.222.0.60 () {56 vars in 1198 bytes} [Wed Sep 28 12:08:17 2022] GET /identity/v3/users/1290a1aa40a149388a72943a7b2dc68e => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:19.270628 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203148bc-2ab9-44c4-8079-398af200c6e6 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:19.271184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203148bc-2ab9-44c4-8079-398af200c6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:19.271184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203148bc-2ab9-44c4-8079-398af200c6e6 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:19.272487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 5/10] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:19.277645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0f3c0-4762-4be6-b722-fe27ad684a16 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:19.278082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0f3c0-4762-4be6-b722-fe27ad684a16 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:19.278178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a0f3c0-4762-4be6-b722-fe27ad684a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:19.296288 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-70a0f3c0-4762-4be6-b722-fe27ad684a16 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:19.297491 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-70a0f3c0-4762-4be6-b722-fe27ad684a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:19.314938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcd32c-fe13-4d28-977e-859aed03525b None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:19.315303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcd32c-fe13-4d28-977e-859aed03525b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:19.315646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcd32c-fe13-4d28-977e-859aed03525b None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:19.317790 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 6/11] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:08:19.322814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ed0c7-bb05-4fbf-a881-4f5583aae08f None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:19.323097 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ed0c7-bb05-4fbf-a881-4f5583aae08f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:19.323372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ed0c7-bb05-4fbf-a881-4f5583aae08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:19.365099 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG passlib.handlers.bcrypt [None req-907ed0c7-bb05-4fbf-a881-4f5583aae08f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81384) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 28 12:08:19.365518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG passlib.handlers.bcrypt [None req-907ed0c7-bb05-4fbf-a881-4f5583aae08f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81384) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 28 12:08:19.381545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 6/12] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:08:19.414895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-907ed0c7-bb05-4fbf-a881-4f5583aae08f None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:19.416479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-907ed0c7-bb05-4fbf-a881-4f5583aae08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:19.515042 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 7/13] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:08:19.647720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0f3c0-4762-4be6-b722-fe27ad684a16 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:19.671789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:19.752126 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a0f3c0-4762-4be6-b722-fe27ad684a16 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:19.783913 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ed0c7-bb05-4fbf-a881-4f5583aae08f None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:19.784194 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2456236-d47c-4628-a16a-1de5446c6eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:19.785408 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2456236-d47c-4628-a16a-1de5446c6eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:19.785695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2456236-d47c-4628-a16a-1de5446c6eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:19.785968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2456236-d47c-4628-a16a-1de5446c6eb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:19.797568 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2456236-d47c-4628-a16a-1de5446c6eb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:19.806527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-e2456236-d47c-4628-a16a-1de5446c6eb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:08:19.806527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:19.806527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:19.806527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:19.806527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:19.806527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:19.806527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:19.806527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:19.806527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:19.806527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:19.806527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:19.806527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:19.806527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:19.806527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:19.806527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:19.806527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:19.807645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:19.807645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:19.807645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:19.807645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:19.807645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:19.807645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:19.807645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:19.807645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:19.807645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:19.807645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:19.807645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:19.807645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:19.807645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:19.807645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:19.807645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:19.807645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:19.808498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:19.808498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:19.808498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:19.808498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:19.808498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:19.808498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:19.808498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:19.808498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:19.808498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:19.808498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:19.808498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:08:19.808498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:08:19.808498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 7/14] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:08:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:19.813323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2456236-d47c-4628-a16a-1de5446c6eb7 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:19.885895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ed0c7-bb05-4fbf-a881-4f5583aae08f None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:19.888195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:19.894617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:19.913887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2456236-d47c-4628-a16a-1de5446c6eb7 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:19.925247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ae059-8874-43a6-bb97-9b4cb73d807d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:19.926456 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ae059-8874-43a6-bb97-9b4cb73d807d None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:19.926743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ae059-8874-43a6-bb97-9b4cb73d807d None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:19.927017 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ae059-8874-43a6-bb97-9b4cb73d807d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:19.946816 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b7ec8-45f1-457c-8b36-f385ffd799f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:19.947410 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b7ec8-45f1-457c-8b36-f385ffd799f0 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:19.947687 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b7ec8-45f1-457c-8b36-f385ffd799f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:19.947958 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b7ec8-45f1-457c-8b36-f385ffd799f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:19.948617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b7ec8-45f1-457c-8b36-f385ffd799f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:19.948919 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b7ec8-45f1-457c-8b36-f385ffd799f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:19.950331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b7ec8-45f1-457c-8b36-f385ffd799f0 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:19.962276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 8/15] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:19.967885 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b7ec8-45f1-457c-8b36-f385ffd799f0 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:19.993619 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ae059-8874-43a6-bb97-9b4cb73d807d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:19.995154 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ae059-8874-43a6-bb97-9b4cb73d807d None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:20.065167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b7ec8-45f1-457c-8b36-f385ffd799f0 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:20.090905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 8/16] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:08:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 12:08:20.096724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ae059-8874-43a6-bb97-9b4cb73d807d None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:20.100236 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:20.102354 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750534c-20e0-4bf6-8b72-3c7033f2412c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:20.102998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750534c-20e0-4bf6-8b72-3c7033f2412c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:20.103285 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750534c-20e0-4bf6-8b72-3c7033f2412c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:20.103557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750534c-20e0-4bf6-8b72-3c7033f2412c None admin] PATH_INFO: `/v3/users/1290a1aa40a149388a72943a7b2dc68e` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:20.104701 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750534c-20e0-4bf6-8b72-3c7033f2412c None admin] RBAC: Authorizing `identity:get_user(user_id=1290a1aa40a149388a72943a7b2dc68e)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:20.106182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750534c-20e0-4bf6-8b72-3c7033f2412c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:20.107736 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 9/17] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:19 2022] GET /identity/v3/users/1290a1aa40a149388a72943a7b2dc68e => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:20.113409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750534c-20e0-4bf6-8b72-3c7033f2412c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:20.127305 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:20.201054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ae059-8874-43a6-bb97-9b4cb73d807d None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:20.212467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750534c-20e0-4bf6-8b72-3c7033f2412c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:20.234934 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06625f0-4387-4a94-bfec-e9b5f18994cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:20.235946 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06625f0-4387-4a94-bfec-e9b5f18994cf None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:20.236486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06625f0-4387-4a94-bfec-e9b5f18994cf None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:20.236761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06625f0-4387-4a94-bfec-e9b5f18994cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:20.246672 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab85e3-5ceb-4f40-b603-2097a2dc80fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:20.247291 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab85e3-5ceb-4f40-b603-2097a2dc80fc None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:20.247571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab85e3-5ceb-4f40-b603-2097a2dc80fc None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:20.247840 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab85e3-5ceb-4f40-b603-2097a2dc80fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:20.248946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab85e3-5ceb-4f40-b603-2097a2dc80fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:20.250423 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab85e3-5ceb-4f40-b603-2097a2dc80fc None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:20.251931 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 10/18] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:08:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:20.255932 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06625f0-4387-4a94-bfec-e9b5f18994cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:20.257427 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06625f0-4387-4a94-bfec-e9b5f18994cf None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:20.258257 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab85e3-5ceb-4f40-b603-2097a2dc80fc None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:20.276011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-f06625f0-4387-4a94-bfec-e9b5f18994cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:08:20.276011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:20.276011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:20.276011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:20.276011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:20.276011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:20.276011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:20.276011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:20.276011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:20.276011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:20.276011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:20.276011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:20.276011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:08:20.276011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:08:20.276011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:08:20.276011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:08:20.277580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:20.277580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:20.277580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:20.277580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:20.277580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:20.277580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:20.277580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:20.277580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:20.277580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:20.277580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:20.277580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:20.277580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:20.277580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:20.277580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:20.277580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:20.277580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:20.277580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:08:20.278664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:08:20.278664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:08:20.278664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:08:20.278664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 9/19] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:08:20.282772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06625f0-4387-4a94-bfec-e9b5f18994cf None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:20.337989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:20.341028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:20.359743 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab85e3-5ceb-4f40-b603-2097a2dc80fc None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:20.383927 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06625f0-4387-4a94-bfec-e9b5f18994cf None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:20.393545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7bcb66-28dc-4d27-b1fe-be75c0f41636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:20.394150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7bcb66-28dc-4d27-b1fe-be75c0f41636 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:20.395024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7bcb66-28dc-4d27-b1fe-be75c0f41636 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:20.395409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7bcb66-28dc-4d27-b1fe-be75c0f41636 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:20.396132 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7bcb66-28dc-4d27-b1fe-be75c0f41636 None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.domain.id=default, user.id=1290a1aa40a149388a72943a7b2dc68e) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:20.396438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7bcb66-28dc-4d27-b1fe-be75c0f41636 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:20.397892 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7bcb66-28dc-4d27-b1fe-be75c0f41636 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:20.404330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 11/20] 10.222.0.60 () {56 vars in 1322 bytes} [Wed Sep 28 12:08:20 2022] GET /identity/v3/role_assignments?user.id=1290a1aa40a149388a72943a7b2dc68e&scope.domain.id=default&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:20.417222 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c16ba-090b-466d-9de5-e493f8b6c297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:20.417852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c16ba-090b-466d-9de5-e493f8b6c297 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:20.418131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c16ba-090b-466d-9de5-e493f8b6c297 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:20.418404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c16ba-090b-466d-9de5-e493f8b6c297 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:20.419080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c16ba-090b-466d-9de5-e493f8b6c297 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:20.419362 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c16ba-090b-466d-9de5-e493f8b6c297 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:20.420800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c16ba-090b-466d-9de5-e493f8b6c297 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:20.440258 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 10/21] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:22.807157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed11a4-3ffe-4936-89da-65a61d0c4228 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:22.807682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed11a4-3ffe-4936-89da-65a61d0c4228 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:22.807995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed11a4-3ffe-4936-89da-65a61d0c4228 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:22.809657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 12/22] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:22.814660 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ebe37-5755-437b-9cba-a4d0e9068245 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:22.815070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ebe37-5755-437b-9cba-a4d0e9068245 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:22.815230 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ebe37-5755-437b-9cba-a4d0e9068245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:22.850168 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-608ebe37-5755-437b-9cba-a4d0e9068245 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:22.851390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-608ebe37-5755-437b-9cba-a4d0e9068245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:22.966480 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 11/23] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:08:23.040118 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c889be-e93d-4610-bfd8-6d64abed27ab None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:23.040463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c889be-e93d-4610-bfd8-6d64abed27ab None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:23.040684 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c889be-e93d-4610-bfd8-6d64abed27ab None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:23.042054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 13/24] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:23.047411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a3de5-69ed-47cd-a69f-b7eb6c856ad9 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:23.047704 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a3de5-69ed-47cd-a69f-b7eb6c856ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:23.047981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a3de5-69ed-47cd-a69f-b7eb6c856ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:23.078377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-0a6a3de5-69ed-47cd-a69f-b7eb6c856ad9 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:23.079527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-0a6a3de5-69ed-47cd-a69f-b7eb6c856ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:23.136712 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:23.172461 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 12/25] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:23.335387 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc7042-b516-4873-b7fb-8f7cb73f74d6 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:23.335639 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc7042-b516-4873-b7fb-8f7cb73f74d6 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:23.336192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc7042-b516-4873-b7fb-8f7cb73f74d6 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:23.338096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 14/26] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:23.343666 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb40acf-c9b9-4b7e-b42b-e420f1754c45 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:23.344199 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb40acf-c9b9-4b7e-b42b-e420f1754c45 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:23.344674 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb40acf-c9b9-4b7e-b42b-e420f1754c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:23.374195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-2eb40acf-c9b9-4b7e-b42b-e420f1754c45 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:23.375431 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc7042-b516-4873-b7fb-8f7cb73f74d6 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:23.385881 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2eb40acf-c9b9-4b7e-b42b-e420f1754c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:23.567196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 13/27] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:08:23.591299 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ed540-febb-4e6e-8e3e-7230332a4ed9 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:23.591916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ed540-febb-4e6e-8e3e-7230332a4ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:23.592400 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ed540-febb-4e6e-8e3e-7230332a4ed9 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:23.593962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 14/28] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:08:23.598449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ed540-febb-4e6e-8e3e-7230332a4ed9 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:23.607515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc7042-b516-4873-b7fb-8f7cb73f74d6 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:23.677259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b3877-611d-4849-90e3-4fc7906b5ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:23.678252 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b3877-611d-4849-90e3-4fc7906b5ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:23.678734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b3877-611d-4849-90e3-4fc7906b5ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:23.679228 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b3877-611d-4849-90e3-4fc7906b5ca3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:23.699846 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b3877-611d-4849-90e3-4fc7906b5ca3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:23.705565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-731b3877-611d-4849-90e3-4fc7906b5ca3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:08:23.705565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:23.705565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:23.705565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:23.705565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:23.705565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:23.705565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:23.705565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:23.705565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:23.705565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:23.705565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:23.705565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:23.705565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:23.705565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:23.705565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:23.705565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:23.706420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:23.706420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:23.706420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:23.706420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:23.706420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:23.706420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:23.706420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:23.706420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:23.706420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:23.706420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:23.706420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:23.706420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:23.706420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:23.706420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:23.706420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:23.706420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:23.707253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:23.707253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:23.707253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:23.707253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:23.707253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:23.707253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:23.707253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:23.707253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:23.707253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:23.707253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:23.707253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:08:23.707253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:08:23.710291 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 15/29] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:08:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:23.713060 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025cf75-9481-42fb-bc98-9f4402b1b878 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:23.714327 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025cf75-9481-42fb-bc98-9f4402b1b878 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:23.715351 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025cf75-9481-42fb-bc98-9f4402b1b878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:23.735922 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-7025cf75-9481-42fb-bc98-9f4402b1b878 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:23.737582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:23.742157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7025cf75-9481-42fb-bc98-9f4402b1b878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:23.785824 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ed540-febb-4e6e-8e3e-7230332a4ed9 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:23.796119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:23.854816 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3f6d6-cb24-4048-9e7c-3532fa321fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:23.855408 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3f6d6-cb24-4048-9e7c-3532fa321fdc None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:23.855692 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3f6d6-cb24-4048-9e7c-3532fa321fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:23.855966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3f6d6-cb24-4048-9e7c-3532fa321fdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:23.876611 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 16/30] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:23.877505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025cf75-9481-42fb-bc98-9f4402b1b878 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:23.878690 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3f6d6-cb24-4048-9e7c-3532fa321fdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:23.881372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3f6d6-cb24-4048-9e7c-3532fa321fdc None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:23.907915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-2eb3f6d6-cb24-4048-9e7c-3532fa321fdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:08:23.907915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:23.907915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:23.907915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:23.907915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:23.907915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:23.907915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:23.907915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:23.907915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:23.907915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:23.907915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:23.907915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:23.907915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:08:23.907915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:08:23.907915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:08:23.907915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:08:23.908877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:23.908877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:23.908877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:23.908877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:23.908877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:23.908877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:23.908877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:23.908877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:23.908877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:23.908877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:23.908877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:23.908877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:23.908877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:23.908877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:23.908877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:23.908877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:23.908877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:08:23.910298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:08:23.910298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:08:23.910298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:08:23.912313 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 15/31] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:23.913306 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3f6d6-cb24-4048-9e7c-3532fa321fdc None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:23.999730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025cf75-9481-42fb-bc98-9f4402b1b878 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:24.022912 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3f6d6-cb24-4048-9e7c-3532fa321fdc None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:24.036107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf29bf-52a7-4ad6-9154-190250a93eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:24.036723 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf29bf-52a7-4ad6-9154-190250a93eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:24.037010 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf29bf-52a7-4ad6-9154-190250a93eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:24.037336 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf29bf-52a7-4ad6-9154-190250a93eb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:24.038002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf29bf-52a7-4ad6-9154-190250a93eb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:24.038304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf29bf-52a7-4ad6-9154-190250a93eb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:24.039940 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf29bf-52a7-4ad6-9154-190250a93eb0 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:24.041498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:24.049541 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 17/32] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:24.050671 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf29bf-52a7-4ad6-9154-190250a93eb0 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:24.062056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd5a5f-a87d-4aa1-98c9-3ccfb7826d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:24.064428 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd5a5f-a87d-4aa1-98c9-3ccfb7826d62 None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:24.064935 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd5a5f-a87d-4aa1-98c9-3ccfb7826d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:24.065431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd5a5f-a87d-4aa1-98c9-3ccfb7826d62 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:24.066888 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd5a5f-a87d-4aa1-98c9-3ccfb7826d62 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:24.068533 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd5a5f-a87d-4aa1-98c9-3ccfb7826d62 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:24.157855 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf29bf-52a7-4ad6-9154-190250a93eb0 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:24.232097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5df04-21ec-47e1-8025-70b14aea7d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:24.232712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5df04-21ec-47e1-8025-70b14aea7d1a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:24.232992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5df04-21ec-47e1-8025-70b14aea7d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:24.234850 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5df04-21ec-47e1-8025-70b14aea7d1a None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:24.235570 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5df04-21ec-47e1-8025-70b14aea7d1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:24.235882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5df04-21ec-47e1-8025-70b14aea7d1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:24.241476 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5df04-21ec-47e1-8025-70b14aea7d1a None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:24.250843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 16/33] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Sep 28 12:08:23 2022] POST /identity/v3/roles => generated 219 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:08:24.252070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd5a5f-a87d-4aa1-98c9-3ccfb7826d62 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:24.290954 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 18/34] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:24.297038 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5df04-21ec-47e1-8025-70b14aea7d1a None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:24.308235 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:24.404708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd5a5f-a87d-4aa1-98c9-3ccfb7826d62 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:24.407661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:24.460954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40f0fc-a8c8-40f1-b5e5-a285b7860eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:24.461783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40f0fc-a8c8-40f1-b5e5-a285b7860eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:24.462068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40f0fc-a8c8-40f1-b5e5-a285b7860eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:24.462345 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40f0fc-a8c8-40f1-b5e5-a285b7860eb0 None admin] PATH_INFO: `/v3/users/1290a1aa40a149388a72943a7b2dc68e` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:24.464765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e40f0fc-a8c8-40f1-b5e5-a285b7860eb0 None admin] RBAC: Authorizing `identity:get_user(user_id=1290a1aa40a149388a72943a7b2dc68e)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:24.466385 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e40f0fc-a8c8-40f1-b5e5-a285b7860eb0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:24.469482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 17/35] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:24 2022] GET /identity/v3/users/1290a1aa40a149388a72943a7b2dc68e => generated 247 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:24.473858 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40f0fc-a8c8-40f1-b5e5-a285b7860eb0 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:24.504726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5df04-21ec-47e1-8025-70b14aea7d1a None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:24.549799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14a0e3-7d9a-40d5-a301-a53a9bf8e9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:24.552025 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14a0e3-7d9a-40d5-a301-a53a9bf8e9b7 None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:24.552319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14a0e3-7d9a-40d5-a301-a53a9bf8e9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:24.552599 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14a0e3-7d9a-40d5-a301-a53a9bf8e9b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:24.553322 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14a0e3-7d9a-40d5-a301-a53a9bf8e9b7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:24.554782 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14a0e3-7d9a-40d5-a301-a53a9bf8e9b7 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:24.615559 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:24.643848 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40f0fc-a8c8-40f1-b5e5-a285b7860eb0 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:24.691867 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f926b-00a1-48d8-93a2-bfaacf1f5c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:24.692539 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f926b-00a1-48d8-93a2-bfaacf1f5c14 None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:24.692834 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f926b-00a1-48d8-93a2-bfaacf1f5c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:24.693110 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f926b-00a1-48d8-93a2-bfaacf1f5c14 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:24.693815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f926b-00a1-48d8-93a2-bfaacf1f5c14 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:24.695248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f926b-00a1-48d8-93a2-bfaacf1f5c14 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:24.721192 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 18/36] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Sep 28 12:08:24 2022] POST /identity/v3/roles => generated 225 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:08:24.730665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f926b-00a1-48d8-93a2-bfaacf1f5c14 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:24.736365 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 19/37] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 12:08:24 2022] POST /identity/v3/projects => generated 301 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:08:24.812012 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316e66c-1b86-4c0c-b69d-52885939f82e None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:24.812338 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316e66c-1b86-4c0c-b69d-52885939f82e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:24.812629 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316e66c-1b86-4c0c-b69d-52885939f82e None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:24.814615 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 20/38] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:24.820886 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da40a4-53b1-4548-a78d-8f6e4dbb768c None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:24.821197 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da40a4-53b1-4548-a78d-8f6e4dbb768c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:24.821409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da40a4-53b1-4548-a78d-8f6e4dbb768c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:24.850775 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-78da40a4-53b1-4548-a78d-8f6e4dbb768c None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:24.852407 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-78da40a4-53b1-4548-a78d-8f6e4dbb768c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:24.866726 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f926b-00a1-48d8-93a2-bfaacf1f5c14 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:24.914094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683eb5ee-5881-4c2b-9785-8bf171995679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:24.915167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683eb5ee-5881-4c2b-9785-8bf171995679 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:24.915669 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683eb5ee-5881-4c2b-9785-8bf171995679 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:24.916159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683eb5ee-5881-4c2b-9785-8bf171995679 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:24.918638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683eb5ee-5881-4c2b-9785-8bf171995679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:24.920573 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683eb5ee-5881-4c2b-9785-8bf171995679 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:24.923787 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 19/39] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:08:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:24.926001 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccfdb6-148b-42d7-b5e0-78074bf89a61 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:24.926507 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccfdb6-148b-42d7-b5e0-78074bf89a61 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:24.926986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccfdb6-148b-42d7-b5e0-78074bf89a61 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:24.928595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 20/40] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:24.934728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccfdb6-148b-42d7-b5e0-78074bf89a61 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:24.949699 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 21/41] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:24.951703 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6344e-54a4-4a20-a225-eae3f36a5fc5 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:24.952209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6344e-54a4-4a20-a225-eae3f36a5fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:24.952735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6344e-54a4-4a20-a225-eae3f36a5fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:24.964302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:24.976905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-30e6344e-54a4-4a20-a225-eae3f36a5fc5 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:24.978092 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-30e6344e-54a4-4a20-a225-eae3f36a5fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:25.027976 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:25.081167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 22/42] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:25.086054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccfdb6-148b-42d7-b5e0-78074bf89a61 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:25.123296 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcb303-72dd-4de8-81a0-dc962d78d0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:25.123964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcb303-72dd-4de8-81a0-dc962d78d0fb None admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:25.124249 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcb303-72dd-4de8-81a0-dc962d78d0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:25.124524 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcb303-72dd-4de8-81a0-dc962d78d0fb None admin] PATH_INFO: `/v3/domains/default/users/1290a1aa40a149388a72943a7b2dc68e/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:25.126521 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcb303-72dd-4de8-81a0-dc962d78d0fb None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1290a1aa40a149388a72943a7b2dc68e, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:25.127961 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcb303-72dd-4de8-81a0-dc962d78d0fb None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:25.143268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 21/43] 10.222.0.60 () {58 vars in 1361 bytes} [Wed Sep 28 12:08:24 2022] PUT /identity/v3/domains/default/users/1290a1aa40a149388a72943a7b2dc68e/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:08:25.240653 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6344e-54a4-4a20-a225-eae3f36a5fc5 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:25.350189 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcb303-72dd-4de8-81a0-dc962d78d0fb None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:25.352671 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6344e-54a4-4a20-a225-eae3f36a5fc5 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:25.438442 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:25.467306 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcb303-72dd-4de8-81a0-dc962d78d0fb None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:25.494340 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad3067-93e6-4ece-80c8-885c7e486d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:25.494881 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad3067-93e6-4ece-80c8-885c7e486d8f None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:25.495169 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad3067-93e6-4ece-80c8-885c7e486d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:25.495454 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad3067-93e6-4ece-80c8-885c7e486d8f None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:25.496144 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad3067-93e6-4ece-80c8-885c7e486d8f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:25.497701 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad3067-93e6-4ece-80c8-885c7e486d8f None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:25.505286 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c601289-a621-4816-bd87-d64c6a43e8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:25.505967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c601289-a621-4816-bd87-d64c6a43e8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:25.506276 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c601289-a621-4816-bd87-d64c6a43e8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:25.506593 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c601289-a621-4816-bd87-d64c6a43e8b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:25.507679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c601289-a621-4816-bd87-d64c6a43e8b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:25.509417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c601289-a621-4816-bd87-d64c6a43e8b9 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:25.513181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 22/44] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:08:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:25.514189 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 23/45] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Sep 28 12:08:25 2022] POST /identity/v3/roles => generated 223 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:08:25.516407 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c601289-a621-4816-bd87-d64c6a43e8b9 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:25.630135 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c601289-a621-4816-bd87-d64c6a43e8b9 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:25.668494 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60285a-dbdf-4cb2-bf70-d3cbf887ef4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:25.669124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60285a-dbdf-4cb2-bf70-d3cbf887ef4b None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:25.669431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60285a-dbdf-4cb2-bf70-d3cbf887ef4b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:25.669705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60285a-dbdf-4cb2-bf70-d3cbf887ef4b None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:25.670359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60285a-dbdf-4cb2-bf70-d3cbf887ef4b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:25.671872 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60285a-dbdf-4cb2-bf70-d3cbf887ef4b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:25.689804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:25.704379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 23/46] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 12:08:25 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:08:27.357717 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e3a67-be7d-4d7d-b6af-f0c139c72c3a None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:27.358260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e3a67-be7d-4d7d-b6af-f0c139c72c3a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:27.358260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e3a67-be7d-4d7d-b6af-f0c139c72c3a None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:27.359611 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 24/47] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:27.366369 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff531d7b-7ba0-4ca7-9d02-21064628636d None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:27.366803 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff531d7b-7ba0-4ca7-9d02-21064628636d None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:27.366940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff531d7b-7ba0-4ca7-9d02-21064628636d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:27.385730 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-ff531d7b-7ba0-4ca7-9d02-21064628636d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:27.387298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-ff531d7b-7ba0-4ca7-9d02-21064628636d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:27.452649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 24/48] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:27.721084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e3a67-be7d-4d7d-b6af-f0c139c72c3a None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:27.748922 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:27.828478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e3a67-be7d-4d7d-b6af-f0c139c72c3a None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:27.862022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a6174-4ab5-48a9-894e-28c4b0c35a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:27.862677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a6174-4ab5-48a9-894e-28c4b0c35a4b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:27.862959 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a6174-4ab5-48a9-894e-28c4b0c35a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:27.863296 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a6174-4ab5-48a9-894e-28c4b0c35a4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:27.873154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a6174-4ab5-48a9-894e-28c4b0c35a4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:27.878053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-9a3a6174-4ab5-48a9-894e-28c4b0c35a4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:08:27.878053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:27.878053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:27.878053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:27.878053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:27.878053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:27.878053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:27.878053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:27.878053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:27.878053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:27.878053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:27.878053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:27.878053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:27.878053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:27.878053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:27.878053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:27.879624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:27.879624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:27.879624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:27.879624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:27.879624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:27.879624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:27.879624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:27.879624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:27.879624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:27.879624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:27.879624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:27.879624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:27.879624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:27.879624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:27.879624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:27.879624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:27.880373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:27.880373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:27.880373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:27.880373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:27.880373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:27.880373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:27.880373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:27.880373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:27.880373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:27.880373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:27.880373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:08:27.880373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:08:27.880373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 25/49] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:08:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:27.885800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff531d7b-7ba0-4ca7-9d02-21064628636d None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:27.900570 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab1e94-d372-438c-99be-dd8f9bc2d65d None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:27.900856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab1e94-d372-438c-99be-dd8f9bc2d65d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:27.901153 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab1e94-d372-438c-99be-dd8f9bc2d65d None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:27.902607 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 26/50] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:08:27.907589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66f04f-f0db-4ac1-b861-1be43949f564 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:27.907869 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66f04f-f0db-4ac1-b861-1be43949f564 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:27.908142 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d66f04f-f0db-4ac1-b861-1be43949f564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:27.925996 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-7d66f04f-f0db-4ac1-b861-1be43949f564 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:27.927175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7d66f04f-f0db-4ac1-b861-1be43949f564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:27.984042 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff531d7b-7ba0-4ca7-9d02-21064628636d None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:27.993694 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 27/51] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:08:28.010140 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:28.021743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f79fd-fa5e-45ac-95ba-dc78c33bd088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:28.022445 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f79fd-fa5e-45ac-95ba-dc78c33bd088 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:28.022732 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f79fd-fa5e-45ac-95ba-dc78c33bd088 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:28.023009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f79fd-fa5e-45ac-95ba-dc78c33bd088 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:28.023665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f79fd-fa5e-45ac-95ba-dc78c33bd088 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:28.023970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f79fd-fa5e-45ac-95ba-dc78c33bd088 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:28.025379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f79fd-fa5e-45ac-95ba-dc78c33bd088 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:28.036821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 25/52] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:28.042441 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66f04f-f0db-4ac1-b861-1be43949f564 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:28.061816 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:28.142466 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d66f04f-f0db-4ac1-b861-1be43949f564 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:28.179970 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d3e40-1dcd-4e59-a52f-181841e3e3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:28.180612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d3e40-1dcd-4e59-a52f-181841e3e3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:28.180897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d3e40-1dcd-4e59-a52f-181841e3e3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:28.181199 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d3e40-1dcd-4e59-a52f-181841e3e3f6 None admin] PATH_INFO: `/v3/users/1290a1aa40a149388a72943a7b2dc68e` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:28.182333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d3e40-1dcd-4e59-a52f-181841e3e3f6 None admin] RBAC: Authorizing `identity:get_user(user_id=1290a1aa40a149388a72943a7b2dc68e)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:28.183784 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273d3e40-1dcd-4e59-a52f-181841e3e3f6 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:28.185537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 28/53] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:28 2022] GET /identity/v3/users/1290a1aa40a149388a72943a7b2dc68e => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:28.190782 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f79fd-fa5e-45ac-95ba-dc78c33bd088 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:28.258104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d3e40-1dcd-4e59-a52f-181841e3e3f6 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:28.347692 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:28.366066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d3e40-1dcd-4e59-a52f-181841e3e3f6 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:28.397521 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f79fd-fa5e-45ac-95ba-dc78c33bd088 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:28.400622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959ffc5-c1b9-4e09-afe4-eadcb5555ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:28.401290 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959ffc5-c1b9-4e09-afe4-eadcb5555ebf None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:28.401576 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959ffc5-c1b9-4e09-afe4-eadcb5555ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:28.401851 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7959ffc5-c1b9-4e09-afe4-eadcb5555ebf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:28.402949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7959ffc5-c1b9-4e09-afe4-eadcb5555ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:28.404399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7959ffc5-c1b9-4e09-afe4-eadcb5555ebf None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:28.405995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 29/54] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:08:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:28.412010 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959ffc5-c1b9-4e09-afe4-eadcb5555ebf None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:28.439284 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd969cd-a808-4934-98c3-079d2d10ca90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:28.439921 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd969cd-a808-4934-98c3-079d2d10ca90 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:28.440202 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd969cd-a808-4934-98c3-079d2d10ca90 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:28.440479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd969cd-a808-4934-98c3-079d2d10ca90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:28.441617 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd969cd-a808-4934-98c3-079d2d10ca90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:28.443061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd969cd-a808-4934-98c3-079d2d10ca90 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:28.444617 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 26/55] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:08:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:28.450324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd969cd-a808-4934-98c3-079d2d10ca90 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:28.503797 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:28.512950 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7959ffc5-c1b9-4e09-afe4-eadcb5555ebf None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:28.547348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd969cd-a808-4934-98c3-079d2d10ca90 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:28.547911 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a19743-380f-490a-9315-ee7e5a9f10bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:28.548497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a19743-380f-490a-9315-ee7e5a9f10bb None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:28.548780 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a19743-380f-490a-9315-ee7e5a9f10bb None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:28.549093 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a19743-380f-490a-9315-ee7e5a9f10bb None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:28.549758 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a19743-380f-490a-9315-ee7e5a9f10bb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:28.552004 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a19743-380f-490a-9315-ee7e5a9f10bb None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:28.577865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 30/56] 10.222.0.60 () {60 vars in 1140 bytes} [Wed Sep 28 12:08:28 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:08:28.581324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e57ddc5-8967-46a4-b71f-18dca8659726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:28.581993 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57ddc5-8967-46a4-b71f-18dca8659726 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:28.582348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57ddc5-8967-46a4-b71f-18dca8659726 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:28.582625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e57ddc5-8967-46a4-b71f-18dca8659726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:28.583363 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57ddc5-8967-46a4-b71f-18dca8659726 None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.domain.id=default, user.id=1290a1aa40a149388a72943a7b2dc68e) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:28.583673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57ddc5-8967-46a4-b71f-18dca8659726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:28.585158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e57ddc5-8967-46a4-b71f-18dca8659726 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:28.594773 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 27/57] 10.222.0.60 () {56 vars in 1322 bytes} [Wed Sep 28 12:08:28 2022] GET /identity/v3/role_assignments?user.id=1290a1aa40a149388a72943a7b2dc68e&scope.domain.id=default&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:30.732815 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8da68-f0c8-45fc-b98e-a9a98f901ca2 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:30.733359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8da68-f0c8-45fc-b98e-a9a98f901ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:30.733443 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec8da68-f0c8-45fc-b98e-a9a98f901ca2 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:30.734758 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 31/58] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:30.740326 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4be1d1-0308-4e9b-bbc5-03b25a264822 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:30.740761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4be1d1-0308-4e9b-bbc5-03b25a264822 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:30.740873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4be1d1-0308-4e9b-bbc5-03b25a264822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:30.758990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-eb4be1d1-0308-4e9b-bbc5-03b25a264822 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:30.760163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-eb4be1d1-0308-4e9b-bbc5-03b25a264822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:30.824087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 28/59] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:31.091511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8da68-f0c8-45fc-b98e-a9a98f901ca2 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:31.101000 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:31.196695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec8da68-f0c8-45fc-b98e-a9a98f901ca2 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:31.228997 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c829880-17a0-43c6-a8da-3b33fe022d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:31.229653 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c829880-17a0-43c6-a8da-3b33fe022d7b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:31.229944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c829880-17a0-43c6-a8da-3b33fe022d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:31.230224 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c829880-17a0-43c6-a8da-3b33fe022d7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:31.231340 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c829880-17a0-43c6-a8da-3b33fe022d7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:31.232775 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c829880-17a0-43c6-a8da-3b33fe022d7b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:31.234368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 32/60] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:08:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:31.240761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4be1d1-0308-4e9b-bbc5-03b25a264822 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:31.287646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:31.336654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4be1d1-0308-4e9b-bbc5-03b25a264822 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:31.369351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffaadc7-a79e-4c73-ae8d-c55add04ecab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:31.369895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffaadc7-a79e-4c73-ae8d-c55add04ecab None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:31.370174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffaadc7-a79e-4c73-ae8d-c55add04ecab None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:31.370443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffaadc7-a79e-4c73-ae8d-c55add04ecab None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:31.371072 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffaadc7-a79e-4c73-ae8d-c55add04ecab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:31.372514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffaadc7-a79e-4c73-ae8d-c55add04ecab None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:31.398442 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 29/61] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:08:31 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:08:34.832182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5cb8a-f94e-41f7-bfb9-4e8f9032102e None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:34.833051 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5cb8a-f94e-41f7-bfb9-4e8f9032102e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:34.833503 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5cb8a-f94e-41f7-bfb9-4e8f9032102e None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:34.835450 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 33/62] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:34.841659 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032a709-28db-4c6c-9d8c-c6db539704c5 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:34.842508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032a709-28db-4c6c-9d8c-c6db539704c5 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:34.842934 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032a709-28db-4c6c-9d8c-c6db539704c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:34.870135 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-8032a709-28db-4c6c-9d8c-c6db539704c5 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:34.871597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-8032a709-28db-4c6c-9d8c-c6db539704c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:34.944074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d840977-4954-47ad-9ab7-f4988713c7cb None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:34.944581 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d840977-4954-47ad-9ab7-f4988713c7cb None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:34.944994 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d840977-4954-47ad-9ab7-f4988713c7cb None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:34.946548 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 34/63] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:34.948393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4390653-4e0b-4fa7-891a-1f712b91b980 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:34.948872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4390653-4e0b-4fa7-891a-1f712b91b980 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:34.953186 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4390653-4e0b-4fa7-891a-1f712b91b980 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:34.955049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 35/64] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:34.956750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfee7a-59e3-48d2-ab5b-6e9059ccbb36 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:34.957199 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfee7a-59e3-48d2-ab5b-6e9059ccbb36 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:34.957421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfee7a-59e3-48d2-ab5b-6e9059ccbb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:34.975665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 30/65] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:34.977298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cfe8f-48e7-4691-94db-db9dbed70c8b None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:34.977755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cfe8f-48e7-4691-94db-db9dbed70c8b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:34.978189 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cfe8f-48e7-4691-94db-db9dbed70c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:34.998753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-e25cfe8f-48e7-4691-94db-db9dbed70c8b None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:35.000098 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-f2bfee7a-59e3-48d2-ab5b-6e9059ccbb36 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:35.000457 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e25cfe8f-48e7-4691-94db-db9dbed70c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:35.001460 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f2bfee7a-59e3-48d2-ab5b-6e9059ccbb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:35.031502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:35.076123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 31/66] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:35.135766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 36/67] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:35.139631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dec30-26cc-43bd-af0b-132f4c49a856 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:35.139922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dec30-26cc-43bd-af0b-132f4c49a856 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:35.140203 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dec30-26cc-43bd-af0b-132f4c49a856 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:35.141637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 32/68] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:35.147708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf97f8a-ab27-4961-be9d-d2bce940c821 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:35.148294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf97f8a-ab27-4961-be9d-d2bce940c821 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:35.148747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf97f8a-ab27-4961-be9d-d2bce940c821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:35.165177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:35.177437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-fbf97f8a-ab27-4961-be9d-d2bce940c821 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:35.178665 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-fbf97f8a-ab27-4961-be9d-d2bce940c821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:35.283393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 37/69] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:35.311609 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748dec30-26cc-43bd-af0b-132f4c49a856 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:35.353496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adbaf8-6004-4aa4-98fe-bd03b1146ed4 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:35.353781 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adbaf8-6004-4aa4-98fe-bd03b1146ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:35.354056 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adbaf8-6004-4aa4-98fe-bd03b1146ed4 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:35.355663 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 38/70] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:35.360511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d649d1-2bb8-4142-8bf1-ee7bfc846d10 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:35.360802 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d649d1-2bb8-4142-8bf1-ee7bfc846d10 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:35.361355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d649d1-2bb8-4142-8bf1-ee7bfc846d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:35.395056 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e7d649d1-2bb8-4142-8bf1-ee7bfc846d10 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:35.396183 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e7d649d1-2bb8-4142-8bf1-ee7bfc846d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:35.437064 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748dec30-26cc-43bd-af0b-132f4c49a856 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:35.467842 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 39/71] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:08:35.468957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d649d1-2bb8-4142-8bf1-ee7bfc846d10 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:35.490175 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389582dd-13b9-4f79-91cc-da359ee31b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:35.490802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389582dd-13b9-4f79-91cc-da359ee31b48 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:35.491146 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389582dd-13b9-4f79-91cc-da359ee31b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:35.491421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389582dd-13b9-4f79-91cc-da359ee31b48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:35.506568 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389582dd-13b9-4f79-91cc-da359ee31b48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:35.514597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-389582dd-13b9-4f79-91cc-da359ee31b48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:08:35.514597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:35.514597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:35.514597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:35.514597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:35.514597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:35.514597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:35.514597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:35.514597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:35.514597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:35.514597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:35.514597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:35.514597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:35.514597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:35.514597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:35.514597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:35.515463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:35.515463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:35.515463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:35.515463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:35.515463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:35.515463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:35.515463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:35.515463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:35.515463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:35.515463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:35.515463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:35.515463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:35.515463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:35.515463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:35.515463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:35.515463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:35.516212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:35.516212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:35.516212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:35.516212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:35.516212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:35.516212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:35.516212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:35.516212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:35.516212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:35.516212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:35.516212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:08:35.516212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:08:35.516212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 33/72] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:08:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:35.516837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389582dd-13b9-4f79-91cc-da359ee31b48 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:35.548648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:35.553317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:35.604317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d649d1-2bb8-4142-8bf1-ee7bfc846d10 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:35.633034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389582dd-13b9-4f79-91cc-da359ee31b48 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:35.643792 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4ab03-648c-4f37-8f7b-d1eabce8dcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:35.644691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4ab03-648c-4f37-8f7b-d1eabce8dcc5 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:35.644980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4ab03-648c-4f37-8f7b-d1eabce8dcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:35.645336 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4ab03-648c-4f37-8f7b-d1eabce8dcc5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:35.655669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4ab03-648c-4f37-8f7b-d1eabce8dcc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:35.660619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-37e4ab03-648c-4f37-8f7b-d1eabce8dcc5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:08:35.660619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:35.660619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:35.660619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:35.660619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:35.660619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:35.660619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:35.660619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:35.660619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:35.660619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:35.660619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:35.660619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:35.660619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:35.660619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:35.660619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:35.660619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:35.661527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:35.661527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:35.661527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:35.661527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:35.661527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:35.661527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:35.661527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:35.661527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:35.661527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:35.661527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:35.661527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:35.661527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:35.661527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:35.661527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:35.661527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:35.661527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:35.662536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:35.662536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:35.662536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:35.662536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:35.662536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:35.662536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:35.662536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:35.662536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:35.662536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:35.662536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:35.662536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:08:35.662536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:08:35.662536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 40/73] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:08:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:35.662536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4ab03-648c-4f37-8f7b-d1eabce8dcc5 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:35.676015 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88713ceb-7053-4bad-af4e-21c4770b971e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:35.676755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88713ceb-7053-4bad-af4e-21c4770b971e None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:35.677039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88713ceb-7053-4bad-af4e-21c4770b971e None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:35.677339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88713ceb-7053-4bad-af4e-21c4770b971e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:35.688153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88713ceb-7053-4bad-af4e-21c4770b971e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:35.693804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-88713ceb-7053-4bad-af4e-21c4770b971e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:08:35.693804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:35.693804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:35.693804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:35.693804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:35.693804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:35.693804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:35.693804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:35.693804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:35.693804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:35.693804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:35.693804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:35.693804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:35.693804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:35.693804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:35.693804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:35.694697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:35.694697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:35.694697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:35.694697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:35.694697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:35.694697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:35.694697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:35.694697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:35.694697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:35.694697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:35.694697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:35.694697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:35.694697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:35.694697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:35.694697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:35.694697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:35.695493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:35.695493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:35.695493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:35.695493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:35.695493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:35.695493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:35.695493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:35.695493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:35.695493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:35.695493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:35.695493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:08:35.695493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:08:35.695493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 34/74] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:35.696123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88713ceb-7053-4bad-af4e-21c4770b971e None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:35.774424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4ab03-648c-4f37-8f7b-d1eabce8dcc5 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:35.781783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:35.797422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88713ceb-7053-4bad-af4e-21c4770b971e None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:35.797701 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:35.810297 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc429600-9c0d-4ba3-83b1-53ca44198fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:35.810907 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc429600-9c0d-4ba3-83b1-53ca44198fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:35.811190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc429600-9c0d-4ba3-83b1-53ca44198fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:35.811461 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc429600-9c0d-4ba3-83b1-53ca44198fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:35.812126 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc429600-9c0d-4ba3-83b1-53ca44198fb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:35.812429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc429600-9c0d-4ba3-83b1-53ca44198fb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:35.813872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc429600-9c0d-4ba3-83b1-53ca44198fb4 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:35.822738 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 41/75] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:35.823596 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc429600-9c0d-4ba3-83b1-53ca44198fb4 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:35.833020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67be09a-6b1e-46d1-8797-48346525023d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:35.833697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67be09a-6b1e-46d1-8797-48346525023d None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:35.833982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67be09a-6b1e-46d1-8797-48346525023d None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:35.834262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67be09a-6b1e-46d1-8797-48346525023d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:35.844316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67be09a-6b1e-46d1-8797-48346525023d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:35.850697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-a67be09a-6b1e-46d1-8797-48346525023d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:08:35.850697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:35.850697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:35.850697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:35.850697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:35.850697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:35.850697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:35.850697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:35.850697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:35.850697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:35.850697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:35.850697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:35.850697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:35.850697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:35.850697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:35.850697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:35.851555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:35.851555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:35.851555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:35.851555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:35.851555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:35.851555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:35.851555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:35.851555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:35.851555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:35.851555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:35.851555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:35.851555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:35.851555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:35.851555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:35.851555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:35.851555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:35.852322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:35.852322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:35.852322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:35.852322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:35.852322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:35.852322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:35.852322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:35.852322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:35.852322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:35.852322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:35.852322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:08:35.852322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:08:35.852322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 35/76] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:35.852965 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67be09a-6b1e-46d1-8797-48346525023d None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:35.925930 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc429600-9c0d-4ba3-83b1-53ca44198fb4 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:35.953376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67be09a-6b1e-46d1-8797-48346525023d None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:35.959655 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c1afe-0560-45a1-ac2c-bfc6907f9a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:35.960266 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c1afe-0560-45a1-ac2c-bfc6907f9a7a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:35.960560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c1afe-0560-45a1-ac2c-bfc6907f9a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:35.961241 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c1afe-0560-45a1-ac2c-bfc6907f9a7a None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:35.961909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c1afe-0560-45a1-ac2c-bfc6907f9a7a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:35.962214 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c1afe-0560-45a1-ac2c-bfc6907f9a7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:35.964263 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c1afe-0560-45a1-ac2c-bfc6907f9a7a None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:35.973708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 42/77] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:08:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:35.974511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c1afe-0560-45a1-ac2c-bfc6907f9a7a None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:35.995389 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b1ae1-fabd-49e7-8bc7-45e09d422ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:35.996289 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b1ae1-fabd-49e7-8bc7-45e09d422ccb None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:35.996722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b1ae1-fabd-49e7-8bc7-45e09d422ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:35.997173 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b1ae1-fabd-49e7-8bc7-45e09d422ccb None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:35.998198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b1ae1-fabd-49e7-8bc7-45e09d422ccb None admin] RBAC: Adding query filter params (name=member) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:35.998677 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b1ae1-fabd-49e7-8bc7-45e09d422ccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:36.000866 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b1ae1-fabd-49e7-8bc7-45e09d422ccb None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:36.010983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 36/78] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:08:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:36.012037 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b1ae1-fabd-49e7-8bc7-45e09d422ccb None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:36.022090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:36.030213 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:36.082753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c1afe-0560-45a1-ac2c-bfc6907f9a7a None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:36.116810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64293381-ac32-47f0-994c-a08ffc7ac90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:36.117488 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64293381-ac32-47f0-994c-a08ffc7ac90c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:36.117777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64293381-ac32-47f0-994c-a08ffc7ac90c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:36.118055 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64293381-ac32-47f0-994c-a08ffc7ac90c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:36.118228 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b1ae1-fabd-49e7-8bc7-45e09d422ccb None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:36.119180 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64293381-ac32-47f0-994c-a08ffc7ac90c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:36.120630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64293381-ac32-47f0-994c-a08ffc7ac90c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:36.122224 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 43/79] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:08:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:36.125926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64293381-ac32-47f0-994c-a08ffc7ac90c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:36.152394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8b298-6c77-41e3-92bf-13b27f71d6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:36.153032 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8b298-6c77-41e3-92bf-13b27f71d6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:36.153359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8b298-6c77-41e3-92bf-13b27f71d6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:36.153641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8b298-6c77-41e3-92bf-13b27f71d6c1 None admin] PATH_INFO: `/v3/users/1290a1aa40a149388a72943a7b2dc68e` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:36.154850 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8b298-6c77-41e3-92bf-13b27f71d6c1 None admin] RBAC: Authorizing `identity:get_user(user_id=1290a1aa40a149388a72943a7b2dc68e)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:36.156290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8b298-6c77-41e3-92bf-13b27f71d6c1 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:36.161158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 37/80] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:36 2022] GET /identity/v3/users/1290a1aa40a149388a72943a7b2dc68e => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:36.161158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8b298-6c77-41e3-92bf-13b27f71d6c1 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:36.233415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:36.234135 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64293381-ac32-47f0-994c-a08ffc7ac90c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:36.250923 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:36.258276 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8b298-6c77-41e3-92bf-13b27f71d6c1 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:36.269107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b667b85-f939-42bb-a74c-0be26e2db5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:36.269740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b667b85-f939-42bb-a74c-0be26e2db5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:36.270021 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b667b85-f939-42bb-a74c-0be26e2db5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:36.270294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b667b85-f939-42bb-a74c-0be26e2db5e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:36.270958 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b667b85-f939-42bb-a74c-0be26e2db5e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:36.271258 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b667b85-f939-42bb-a74c-0be26e2db5e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:36.272645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b667b85-f939-42bb-a74c-0be26e2db5e7 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:36.280399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 44/81] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:08:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:36.281197 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b667b85-f939-42bb-a74c-0be26e2db5e7 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:36.292827 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cfb71-ee3d-471f-a66e-b375768ff9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:36.293498 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cfb71-ee3d-471f-a66e-b375768ff9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:36.293780 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cfb71-ee3d-471f-a66e-b375768ff9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:36.294055 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6cfb71-ee3d-471f-a66e-b375768ff9d1 None admin] PATH_INFO: `/v3/users/a45fb43467b9498eb1b647735d0b88b5` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:36.309817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6cfb71-ee3d-471f-a66e-b375768ff9d1 None admin] RBAC: Authorizing `identity:get_user(user_id=a45fb43467b9498eb1b647735d0b88b5)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:36.311279 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6cfb71-ee3d-471f-a66e-b375768ff9d1 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:36.312915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 38/82] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:36 2022] GET /identity/v3/users/a45fb43467b9498eb1b647735d0b88b5 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:36.313686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cfb71-ee3d-471f-a66e-b375768ff9d1 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:36.382840 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b667b85-f939-42bb-a74c-0be26e2db5e7 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:36.414764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6cfb71-ee3d-471f-a66e-b375768ff9d1 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:36.416661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfd3ba-8fbe-4606-ab00-8baa2e0e02b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:36.417358 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfd3ba-8fbe-4606-ab00-8baa2e0e02b5 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:36.417628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfd3ba-8fbe-4606-ab00-8baa2e0e02b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:36.417905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfd3ba-8fbe-4606-ab00-8baa2e0e02b5 None admin] PATH_INFO: `/v3/users/a45fb43467b9498eb1b647735d0b88b5` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:36.419038 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfd3ba-8fbe-4606-ab00-8baa2e0e02b5 None admin] RBAC: Authorizing `identity:get_user(user_id=a45fb43467b9498eb1b647735d0b88b5)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:36.420501 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfd3ba-8fbe-4606-ab00-8baa2e0e02b5 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:36.422287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 45/83] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:36 2022] GET /identity/v3/users/a45fb43467b9498eb1b647735d0b88b5 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:36.422889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfd3ba-8fbe-4606-ab00-8baa2e0e02b5 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:36.447916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:36.453400 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8778d3-070b-443e-9ee5-9a859a4ec5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:36.454017 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8778d3-070b-443e-9ee5-9a859a4ec5a3 None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:36.454406 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8778d3-070b-443e-9ee5-9a859a4ec5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:36.454570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8778d3-070b-443e-9ee5-9a859a4ec5a3 None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:36.455215 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8778d3-070b-443e-9ee5-9a859a4ec5a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:36.457242 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8778d3-070b-443e-9ee5-9a859a4ec5a3 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:36.475542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:36.478259 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 39/84] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:08:36 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:08:36.487410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8778d3-070b-443e-9ee5-9a859a4ec5a3 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:36.537537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfd3ba-8fbe-4606-ab00-8baa2e0e02b5 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:36.581890 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62482e08-8151-4642-a795-a0630d08cffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:36.582646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62482e08-8151-4642-a795-a0630d08cffb None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:36.582971 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62482e08-8151-4642-a795-a0630d08cffb None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:36.583253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62482e08-8151-4642-a795-a0630d08cffb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:36.613627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8778d3-070b-443e-9ee5-9a859a4ec5a3 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:36.655954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7d82a-17a6-44c1-b4d2-3a22228d25db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:36.656170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7d82a-17a6-44c1-b4d2-3a22228d25db None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:36.656406 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7d82a-17a6-44c1-b4d2-3a22228d25db None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:36.656617 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7d82a-17a6-44c1-b4d2-3a22228d25db None admin] PATH_INFO: `/v3/users/a45fb43467b9498eb1b647735d0b88b5` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:36.658290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7d82a-17a6-44c1-b4d2-3a22228d25db None admin] RBAC: Authorizing `identity:get_user(user_id=a45fb43467b9498eb1b647735d0b88b5)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:36.660636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7d82a-17a6-44c1-b4d2-3a22228d25db None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:36.662387 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 40/85] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:36 2022] GET /identity/v3/users/a45fb43467b9498eb1b647735d0b88b5 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:36.662496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62482e08-8151-4642-a795-a0630d08cffb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:36.662571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7d82a-17a6-44c1-b4d2-3a22228d25db None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:36.663599 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62482e08-8151-4642-a795-a0630d08cffb None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:36.721656 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:36.736754 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:36.791869 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 46/86] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:08:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:08:36.793346 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62482e08-8151-4642-a795-a0630d08cffb None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:36.795792 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7d82a-17a6-44c1-b4d2-3a22228d25db None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:36.840603 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0819ea-4ba1-43b0-abb0-05f34fd7e442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:36.841288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0819ea-4ba1-43b0-abb0-05f34fd7e442 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:36.841566 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0819ea-4ba1-43b0-abb0-05f34fd7e442 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:36.841840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0819ea-4ba1-43b0-abb0-05f34fd7e442 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:36.919705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0819ea-4ba1-43b0-abb0-05f34fd7e442 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:36.921268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0819ea-4ba1-43b0-abb0-05f34fd7e442 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:36.921544 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62482e08-8151-4642-a795-a0630d08cffb None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:36.963383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61827c-80f1-4eee-89a1-3ef23109bcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:36.964006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61827c-80f1-4eee-89a1-3ef23109bcf6 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:36.964272 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61827c-80f1-4eee-89a1-3ef23109bcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:36.964550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61827c-80f1-4eee-89a1-3ef23109bcf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:36.981116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:36.984798 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:37.040367 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 41/87] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:08:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:08:37.041281 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0819ea-4ba1-43b0-abb0-05f34fd7e442 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:37.048985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61827c-80f1-4eee-89a1-3ef23109bcf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:37.051299 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61827c-80f1-4eee-89a1-3ef23109bcf6 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:37.155686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0819ea-4ba1-43b0-abb0-05f34fd7e442 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:37.177319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 47/88] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:08:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:08:37.178395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61827c-80f1-4eee-89a1-3ef23109bcf6 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:37.202895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1db17b-f738-4c2e-9a12-9927bcfba1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:37.203648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1db17b-f738-4c2e-9a12-9927bcfba1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:37.204044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1db17b-f738-4c2e-9a12-9927bcfba1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:37.204412 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1db17b-f738-4c2e-9a12-9927bcfba1d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:37.237503 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:37.241268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:37.296113 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61827c-80f1-4eee-89a1-3ef23109bcf6 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:37.305240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1db17b-f738-4c2e-9a12-9927bcfba1d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:37.307349 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1db17b-f738-4c2e-9a12-9927bcfba1d1 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:37.334416 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ab51c-b71b-4854-a3dc-7994d91bd0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:37.335066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ab51c-b71b-4854-a3dc-7994d91bd0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:37.335359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ab51c-b71b-4854-a3dc-7994d91bd0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:37.335642 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ab51c-b71b-4854-a3dc-7994d91bd0a6 None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:37.336793 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ab51c-b71b-4854-a3dc-7994d91bd0a6 None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:37.338377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ab51c-b71b-4854-a3dc-7994d91bd0a6 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:37.340358 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 48/89] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:37 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:37.341170 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ab51c-b71b-4854-a3dc-7994d91bd0a6 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:37.451020 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ab51c-b71b-4854-a3dc-7994d91bd0a6 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:37.452951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 42/90] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:08:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:08:37.454769 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1db17b-f738-4c2e-9a12-9927bcfba1d1 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:37.468256 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:37.491661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761e1a0-5089-4fa3-bd16-404fed4076b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:37.492365 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761e1a0-5089-4fa3-bd16-404fed4076b7 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:37.492656 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761e1a0-5089-4fa3-bd16-404fed4076b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:37.492987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761e1a0-5089-4fa3-bd16-404fed4076b7 None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:37.494220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1761e1a0-5089-4fa3-bd16-404fed4076b7 None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:37.495671 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1761e1a0-5089-4fa3-bd16-404fed4076b7 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:37.497676 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 49/91] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:37 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:37.498550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761e1a0-5089-4fa3-bd16-404fed4076b7 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:37.529037 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:37.585164 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1db17b-f738-4c2e-9a12-9927bcfba1d1 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:37.609056 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761e1a0-5089-4fa3-bd16-404fed4076b7 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:37.628886 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a309557-3485-474b-a478-280a4de12cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:37.629576 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a309557-3485-474b-a478-280a4de12cfb None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:37.629860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a309557-3485-474b-a478-280a4de12cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:37.630323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a309557-3485-474b-a478-280a4de12cfb None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:37.631919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a309557-3485-474b-a478-280a4de12cfb None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:37.633479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a309557-3485-474b-a478-280a4de12cfb None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:37.635909 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 43/92] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:37 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:37.637491 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a309557-3485-474b-a478-280a4de12cfb None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:37.660426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536ab4b-e8e2-4541-ae0e-c2438bf63436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:37.660982 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536ab4b-e8e2-4541-ae0e-c2438bf63436 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:37.661348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536ab4b-e8e2-4541-ae0e-c2438bf63436 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:37.661636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536ab4b-e8e2-4541-ae0e-c2438bf63436 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:37.662377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536ab4b-e8e2-4541-ae0e-c2438bf63436 None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3, user.id=1290a1aa40a149388a72943a7b2dc68e) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:37.662691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536ab4b-e8e2-4541-ae0e-c2438bf63436 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:37.664153 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536ab4b-e8e2-4541-ae0e-c2438bf63436 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:37.669793 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 50/93] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:08:37 2022] GET /identity/v3/role_assignments?user.id=1290a1aa40a149388a72943a7b2dc68e&scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:37.670589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536ab4b-e8e2-4541-ae0e-c2438bf63436 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:37.773117 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:37.842202 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a309557-3485-474b-a478-280a4de12cfb None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:37.884265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536ab4b-e8e2-4541-ae0e-c2438bf63436 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:37.897767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:37.917978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de04801-e49a-4307-84f0-2794a8474a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:37.918637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de04801-e49a-4307-84f0-2794a8474a81 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:37.918924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de04801-e49a-4307-84f0-2794a8474a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:37.919201 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de04801-e49a-4307-84f0-2794a8474a81 None admin] PATH_INFO: `/v3/projects/dee48d9648274dd1a8f35ba629aa41b0` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:37.920478 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de04801-e49a-4307-84f0-2794a8474a81 None admin] RBAC: Authorizing `identity:get_project(project_id=dee48d9648274dd1a8f35ba629aa41b0)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:37.922101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de04801-e49a-4307-84f0-2794a8474a81 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:37.923859 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 44/94] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:37 2022] GET /identity/v3/projects/dee48d9648274dd1a8f35ba629aa41b0 => generated 312 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:37.924828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de04801-e49a-4307-84f0-2794a8474a81 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:37.940063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4018af8-1ad7-4387-9084-a321bc9ac136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:37.940718 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4018af8-1ad7-4387-9084-a321bc9ac136 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:37.941001 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4018af8-1ad7-4387-9084-a321bc9ac136 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:37.941302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4018af8-1ad7-4387-9084-a321bc9ac136 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:37.942035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4018af8-1ad7-4387-9084-a321bc9ac136 None admin] RBAC: Adding query filter params (role.id=8e9576d16f194938b86a1d1e087895bd, scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3, user.id=a45fb43467b9498eb1b647735d0b88b5) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:37.942344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4018af8-1ad7-4387-9084-a321bc9ac136 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:37.943787 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4018af8-1ad7-4387-9084-a321bc9ac136 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:37.957739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 51/95] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:08:37 2022] GET /identity/v3/role_assignments?user.id=a45fb43467b9498eb1b647735d0b88b5&scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3&role.id=8e9576d16f194938b86a1d1e087895bd => generated 259 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:37.958593 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4018af8-1ad7-4387-9084-a321bc9ac136 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:38.143328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de04801-e49a-4307-84f0-2794a8474a81 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:38.171669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4018af8-1ad7-4387-9084-a321bc9ac136 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:38.193911 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:38.234854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01834d-0d2a-4e67-8755-850842d62afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:38.235829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01834d-0d2a-4e67-8755-850842d62afc None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:38.236563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01834d-0d2a-4e67-8755-850842d62afc None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:38.237076 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01834d-0d2a-4e67-8755-850842d62afc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:38.238169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01834d-0d2a-4e67-8755-850842d62afc None admin] RBAC: Adding query filter params (role.id=2201e005d5ef449896a0e68a940678ec, scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3, user.id=a45fb43467b9498eb1b647735d0b88b5) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:38.238704 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01834d-0d2a-4e67-8755-850842d62afc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:38.240360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01834d-0d2a-4e67-8755-850842d62afc None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:38.254172 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 45/96] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:08:37 2022] GET /identity/v3/role_assignments?user.id=a45fb43467b9498eb1b647735d0b88b5&scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3&role.id=2201e005d5ef449896a0e68a940678ec => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:38.255496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b46b3a-b777-4846-9786-d583103a04f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:38.256136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b46b3a-b777-4846-9786-d583103a04f1 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:38.256425 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b46b3a-b777-4846-9786-d583103a04f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:38.256707 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b46b3a-b777-4846-9786-d583103a04f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:38.257489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b46b3a-b777-4846-9786-d583103a04f1 None admin] RBAC: Adding query filter params (role.id=2201e005d5ef449896a0e68a940678ec, scope.project.id=dee48d9648274dd1a8f35ba629aa41b0, user.id=a45fb43467b9498eb1b647735d0b88b5) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:38.257807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b46b3a-b777-4846-9786-d583103a04f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:38.259382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b46b3a-b777-4846-9786-d583103a04f1 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:38.271010 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 52/97] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:08:37 2022] GET /identity/v3/role_assignments?user.id=a45fb43467b9498eb1b647735d0b88b5&scope.project.id=dee48d9648274dd1a8f35ba629aa41b0&role.id=2201e005d5ef449896a0e68a940678ec => generated 259 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:39.083621 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd5514-c295-4197-a3f5-8bedec832e28 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:39.084667 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd5514-c295-4197-a3f5-8bedec832e28 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:39.085175 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd5514-c295-4197-a3f5-8bedec832e28 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:39.086719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 46/98] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:39.099150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0ffab-55e5-4e9a-9f41-47f71f44136c None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:39.099771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0ffab-55e5-4e9a-9f41-47f71f44136c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:39.100246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e0ffab-55e5-4e9a-9f41-47f71f44136c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:39.130420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-50e0ffab-55e5-4e9a-9f41-47f71f44136c None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:39.132014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-50e0ffab-55e5-4e9a-9f41-47f71f44136c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:39.223207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 53/99] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:39.241258 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d117e-a0d4-4fe2-8d8b-461c21db848f None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:39.241574 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d117e-a0d4-4fe2-8d8b-461c21db848f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:39.241827 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d117e-a0d4-4fe2-8d8b-461c21db848f None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:39.243301 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 47/100] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:39.248532 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b48254-0f79-445a-afa3-a00a642e538c None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:39.248828 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b48254-0f79-445a-afa3-a00a642e538c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:39.249112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b48254-0f79-445a-afa3-a00a642e538c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:39.273085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-27b48254-0f79-445a-afa3-a00a642e538c None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:39.274302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-27b48254-0f79-445a-afa3-a00a642e538c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:39.363415 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 54/101] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:39.745439 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d117e-a0d4-4fe2-8d8b-461c21db848f None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:39.767015 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b48254-0f79-445a-afa3-a00a642e538c None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:39.817154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:39.833535 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:39.918143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d117e-a0d4-4fe2-8d8b-461c21db848f None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:39.988093 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1969a-d745-4b4b-8884-0cf512381adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:39.988762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1969a-d745-4b4b-8884-0cf512381adc None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:39.989026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1969a-d745-4b4b-8884-0cf512381adc None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:39.989324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1969a-d745-4b4b-8884-0cf512381adc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:40.003585 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b48254-0f79-445a-afa3-a00a642e538c None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:40.013047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1969a-d745-4b4b-8884-0cf512381adc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:40.019572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-3ad1969a-d745-4b4b-8884-0cf512381adc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:08:40.019572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:40.019572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:40.019572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:40.019572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:40.019572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:40.019572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:40.019572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:40.019572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:40.019572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:40.019572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:40.019572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:40.019572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:40.019572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:40.019572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:40.019572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:40.020430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:40.020430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:40.020430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:40.020430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:40.020430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:40.020430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:40.020430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:40.020430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:40.020430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:40.020430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:40.020430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:40.020430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:40.020430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:40.020430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:40.020430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:40.020430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:40.021225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:40.021225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:40.021225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:40.021225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:40.021225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:40.021225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:40.021225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:40.021225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:40.021225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:40.021225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:40.021225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:08:40.021225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:08:40.021225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 48/102] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:40.026957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1969a-d745-4b4b-8884-0cf512381adc None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:40.133866 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39518879-218c-4d45-a358-c1b566c66d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:40.134535 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39518879-218c-4d45-a358-c1b566c66d61 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:40.134829 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39518879-218c-4d45-a358-c1b566c66d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:40.135108 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39518879-218c-4d45-a358-c1b566c66d61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:40.141826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39518879-218c-4d45-a358-c1b566c66d61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:40.143308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39518879-218c-4d45-a358-c1b566c66d61 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:40.146401 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 55/103] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:08:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:40.152122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39518879-218c-4d45-a358-c1b566c66d61 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:40.194036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1969a-d745-4b4b-8884-0cf512381adc None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:40.225590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:40.249767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f53ab-380d-4e77-b045-93784915eda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:40.250530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f53ab-380d-4e77-b045-93784915eda7 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:40.250817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f53ab-380d-4e77-b045-93784915eda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:40.251089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f53ab-380d-4e77-b045-93784915eda7 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:40.251750 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f53ab-380d-4e77-b045-93784915eda7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:40.252053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f53ab-380d-4e77-b045-93784915eda7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:40.253665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f53ab-380d-4e77-b045-93784915eda7 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:40.262211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 49/104] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:08:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:40.270137 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f53ab-380d-4e77-b045-93784915eda7 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:40.426740 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f53ab-380d-4e77-b045-93784915eda7 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:40.442533 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39518879-218c-4d45-a358-c1b566c66d61 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:40.444163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:40.483172 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccace8e-0bcc-49ae-8a29-bf9bb3ea9fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:40.483817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccace8e-0bcc-49ae-8a29-bf9bb3ea9fab None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:40.484110 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccace8e-0bcc-49ae-8a29-bf9bb3ea9fab None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:40.484389 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccace8e-0bcc-49ae-8a29-bf9bb3ea9fab None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:40.485071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccace8e-0bcc-49ae-8a29-bf9bb3ea9fab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:40.486869 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccace8e-0bcc-49ae-8a29-bf9bb3ea9fab None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:40.490437 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20906c14-a831-41b3-9598-f49a50206b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:40.491107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20906c14-a831-41b3-9598-f49a50206b54 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:40.491395 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20906c14-a831-41b3-9598-f49a50206b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:40.491674 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20906c14-a831-41b3-9598-f49a50206b54 None admin] PATH_INFO: `/v3/users/17f1c3d7516145758cee5edf5ebc5888` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:40.521978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20906c14-a831-41b3-9598-f49a50206b54 None admin] RBAC: Authorizing `identity:get_user(user_id=17f1c3d7516145758cee5edf5ebc5888)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:40.523394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20906c14-a831-41b3-9598-f49a50206b54 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:40.525470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 50/105] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:40 2022] GET /identity/v3/users/17f1c3d7516145758cee5edf5ebc5888 => generated 289 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:40.532151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20906c14-a831-41b3-9598-f49a50206b54 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:40.541063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 56/106] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 12:08:40 2022] POST /identity/v3/projects => generated 302 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:08:40.586911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:40.672705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20906c14-a831-41b3-9598-f49a50206b54 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:40.721830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17c65c-635a-4991-b5e5-3cce390e27fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:40.722204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e63b94-9a17-4802-ba31-c4f2d71582fd None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:40.722204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e63b94-9a17-4802-ba31-c4f2d71582fd None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:40.722672 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e63b94-9a17-4802-ba31-c4f2d71582fd None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:40.723385 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17c65c-635a-4991-b5e5-3cce390e27fd None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:40.723856 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17c65c-635a-4991-b5e5-3cce390e27fd None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:40.724268 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 57/107] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:40.724537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17c65c-635a-4991-b5e5-3cce390e27fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:40.747396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7446b-9a72-4429-ba8e-7dd9638b0b06 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:40.747699 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7446b-9a72-4429-ba8e-7dd9638b0b06 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:40.747973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7446b-9a72-4429-ba8e-7dd9638b0b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:40.786434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-0ff7446b-9a72-4429-ba8e-7dd9638b0b06 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:40.787604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0ff7446b-9a72-4429-ba8e-7dd9638b0b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:40.809912 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17c65c-635a-4991-b5e5-3cce390e27fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:40.812160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17c65c-635a-4991-b5e5-3cce390e27fd None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:40.852124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:40.929729 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 58/108] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:40.930858 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 51/109] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:08:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 12:08:40.943449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7446b-9a72-4429-ba8e-7dd9638b0b06 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:41.066446 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:41.081752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc06712-a679-4f3c-b578-4e3732abc16f None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:41.082372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc06712-a679-4f3c-b578-4e3732abc16f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:41.082823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc06712-a679-4f3c-b578-4e3732abc16f None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:41.084380 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 52/110] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:41.089282 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7446b-9a72-4429-ba8e-7dd9638b0b06 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:41.091558 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307fa1e-714c-43ba-8d81-0c8667f7d5f5 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:41.092053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307fa1e-714c-43ba-8d81-0c8667f7d5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:41.092520 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307fa1e-714c-43ba-8d81-0c8667f7d5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:41.125289 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-1307fa1e-714c-43ba-8d81-0c8667f7d5f5 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:41.126806 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-1307fa1e-714c-43ba-8d81-0c8667f7d5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:41.148740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840a223-db7a-4b7c-bb92-53c3091a1b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:41.148740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840a223-db7a-4b7c-bb92-53c3091a1b0b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:41.149009 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840a223-db7a-4b7c-bb92-53c3091a1b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:41.149402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840a223-db7a-4b7c-bb92-53c3091a1b0b None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:41.154073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840a223-db7a-4b7c-bb92-53c3091a1b0b None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:41.155555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840a223-db7a-4b7c-bb92-53c3091a1b0b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:41.157727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 59/111] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:40 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:41.164242 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840a223-db7a-4b7c-bb92-53c3091a1b0b None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:41.233166 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 53/112] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:41.326310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307fa1e-714c-43ba-8d81-0c8667f7d5f5 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:41.359948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840a223-db7a-4b7c-bb92-53c3091a1b0b None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:41.449510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:41.449914 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24e353-2c8a-4ae8-84a0-5bf892763e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:41.450557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24e353-2c8a-4ae8-84a0-5bf892763e0f None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:41.450841 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24e353-2c8a-4ae8-84a0-5bf892763e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:41.451124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24e353-2c8a-4ae8-84a0-5bf892763e0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:41.451869 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24e353-2c8a-4ae8-84a0-5bf892763e0f None admin] RBAC: Adding query filter params (role.id=edf27bedabd24681b24ec2a72a774a9c, scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3, user.id=17f1c3d7516145758cee5edf5ebc5888) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:41.452183 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24e353-2c8a-4ae8-84a0-5bf892763e0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:41.453884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24e353-2c8a-4ae8-84a0-5bf892763e0f None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:41.467206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 60/113] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:08:41 2022] GET /identity/v3/role_assignments?user.id=17f1c3d7516145758cee5edf5ebc5888&scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3&role.id=edf27bedabd24681b24ec2a72a774a9c => generated 259 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:41.504278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307fa1e-714c-43ba-8d81-0c8667f7d5f5 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:41.559602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a130c-3473-499e-bbda-9db93c7750ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:41.560587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a130c-3473-499e-bbda-9db93c7750ff None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:41.561065 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a130c-3473-499e-bbda-9db93c7750ff None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:41.561582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a130c-3473-499e-bbda-9db93c7750ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:41.575160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a130c-3473-499e-bbda-9db93c7750ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:41.580555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-393a130c-3473-499e-bbda-9db93c7750ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:08:41.580555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:41.580555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:41.580555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:41.580555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:41.580555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:41.580555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:41.580555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:41.580555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:41.580555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:41.580555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:41.580555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:41.580555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:41.580555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:41.580555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:41.580555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:41.582047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:41.582047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:41.582047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:41.582047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:41.582047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:41.582047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:41.582047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:41.582047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:41.582047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:41.582047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:41.582047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:41.582047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:41.582047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:41.582047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:41.582047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:41.582047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:41.582967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:41.582967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:41.582967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:41.582967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:41.582967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:41.582967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:41.582967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:41.582967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:41.582967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:41.582967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:41.582967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:08:41.582967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:08:41.584820 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 54/114] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:08:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:41.591015 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24e353-2c8a-4ae8-84a0-5bf892763e0f None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:41.613432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:41.620545 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a130c-3473-499e-bbda-9db93c7750ff None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:41.788509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a130c-3473-499e-bbda-9db93c7750ff None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:41.803639 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24e353-2c8a-4ae8-84a0-5bf892763e0f None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:41.819428 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:41.829505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860f201-9974-44fc-8a35-5aac195d118a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:41.830154 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860f201-9974-44fc-8a35-5aac195d118a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:41.830443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860f201-9974-44fc-8a35-5aac195d118a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:41.830777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860f201-9974-44fc-8a35-5aac195d118a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:41.851176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860f201-9974-44fc-8a35-5aac195d118a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:41.857169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-f860f201-9974-44fc-8a35-5aac195d118a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:08:41.857169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:41.857169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:41.857169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:41.857169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:41.857169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:41.857169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:41.857169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:41.857169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:41.857169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:41.857169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:41.857169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:41.857169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:41.857169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:41.857169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:41.857169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:41.858004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:41.858004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:41.858004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:41.858004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:41.858004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:41.858004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:41.858004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:41.858004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:41.858004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:41.858004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:41.858004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:41.858004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:41.858004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:41.858004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:41.858004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:41.858004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:41.858772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:41.858772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:41.858772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:41.858772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:41.858772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:41.858772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:41.858772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:41.858772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:41.858772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:41.858772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:41.858772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:08:41.858772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:08:41.858772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 55/115] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:41.870047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860f201-9974-44fc-8a35-5aac195d118a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:41.890187 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bdaef0-d0a3-4b64-a010-38fe37057987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:41.890814 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdaef0-d0a3-4b64-a010-38fe37057987 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:41.891101 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdaef0-d0a3-4b64-a010-38fe37057987 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:41.891381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bdaef0-d0a3-4b64-a010-38fe37057987 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:41.892124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bdaef0-d0a3-4b64-a010-38fe37057987 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:41.892435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bdaef0-d0a3-4b64-a010-38fe37057987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:41.894138 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bdaef0-d0a3-4b64-a010-38fe37057987 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:41.910515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 61/116] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:41.912055 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79300c0-cfdd-4665-9208-849a55e1ab82 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:41.912346 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79300c0-cfdd-4665-9208-849a55e1ab82 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:41.912621 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79300c0-cfdd-4665-9208-849a55e1ab82 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:41.913874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 62/117] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:41.918527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79300c0-cfdd-4665-9208-849a55e1ab82 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:42.014472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860f201-9974-44fc-8a35-5aac195d118a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:42.024109 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:42.054247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a32ee3-dea1-4f38-b7bf-94127000efc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:42.055212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a32ee3-dea1-4f38-b7bf-94127000efc4 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:42.055636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a32ee3-dea1-4f38-b7bf-94127000efc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:42.055745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79300c0-cfdd-4665-9208-849a55e1ab82 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:42.056154 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a32ee3-dea1-4f38-b7bf-94127000efc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:42.056994 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a32ee3-dea1-4f38-b7bf-94127000efc4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:42.057554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a32ee3-dea1-4f38-b7bf-94127000efc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:42.059265 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a32ee3-dea1-4f38-b7bf-94127000efc4 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:42.069719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 56/118] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:08:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:42.076019 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39193b6f-1ec0-455f-a3e7-06e44ffee5c3 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:42.076528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39193b6f-1ec0-455f-a3e7-06e44ffee5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:42.076995 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39193b6f-1ec0-455f-a3e7-06e44ffee5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:42.096121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c458b51-9849-4a09-ac74-3dd23d04368f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:42.097219 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c458b51-9849-4a09-ac74-3dd23d04368f None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:42.097508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c458b51-9849-4a09-ac74-3dd23d04368f None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:42.097785 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c458b51-9849-4a09-ac74-3dd23d04368f None admin] PATH_INFO: `/v3/users/1290a1aa40a149388a72943a7b2dc68e` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:42.098045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-39193b6f-1ec0-455f-a3e7-06e44ffee5c3 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:42.098915 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c458b51-9849-4a09-ac74-3dd23d04368f None admin] RBAC: Authorizing `identity:get_user(user_id=1290a1aa40a149388a72943a7b2dc68e)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:42.099624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-39193b6f-1ec0-455f-a3e7-06e44ffee5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:42.100375 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c458b51-9849-4a09-ac74-3dd23d04368f None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:42.102540 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 63/119] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:41 2022] GET /identity/v3/users/1290a1aa40a149388a72943a7b2dc68e => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:42.105897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715a41e-3745-4d8c-8e7d-43d5ab6c3b2b None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:42.106181 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715a41e-3745-4d8c-8e7d-43d5ab6c3b2b None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:42.106457 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715a41e-3745-4d8c-8e7d-43d5ab6c3b2b None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:42.107669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 64/120] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:42.108366 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715a41e-3745-4d8c-8e7d-43d5ab6c3b2b None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:42.152953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:42.186144 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 57/121] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:42.189928 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39193b6f-1ec0-455f-a3e7-06e44ffee5c3 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:42.262578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715a41e-3745-4d8c-8e7d-43d5ab6c3b2b None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:42.325868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c252b4c-a7b4-4be0-b4fd-95097408d625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:42.326514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c252b4c-a7b4-4be0-b4fd-95097408d625 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:42.326798 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c252b4c-a7b4-4be0-b4fd-95097408d625 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:42.327073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c252b4c-a7b4-4be0-b4fd-95097408d625 None admin] PATH_INFO: `/v3/users/a45fb43467b9498eb1b647735d0b88b5` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:42.329107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c252b4c-a7b4-4be0-b4fd-95097408d625 None admin] RBAC: Authorizing `identity:get_user(user_id=a45fb43467b9498eb1b647735d0b88b5)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:42.330594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c252b4c-a7b4-4be0-b4fd-95097408d625 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:42.332357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 65/122] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:42 2022] GET /identity/v3/users/a45fb43467b9498eb1b647735d0b88b5 => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:42.334490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f5676-3190-407b-bdbc-14bd4fd3fb1d None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:42.334773 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f5676-3190-407b-bdbc-14bd4fd3fb1d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:42.335052 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f5676-3190-407b-bdbc-14bd4fd3fb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:42.368925 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-ea5f5676-3190-407b-bdbc-14bd4fd3fb1d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:42.372393 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39193b6f-1ec0-455f-a3e7-06e44ffee5c3 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:42.374058 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-ea5f5676-3190-407b-bdbc-14bd4fd3fb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:42.391507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:42.435803 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3c1f8-b563-45f0-967a-8f9820028a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:42.436963 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3c1f8-b563-45f0-967a-8f9820028a83 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:42.437659 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3c1f8-b563-45f0-967a-8f9820028a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:42.437943 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3c1f8-b563-45f0-967a-8f9820028a83 None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:42.439567 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3c1f8-b563-45f0-967a-8f9820028a83 None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:42.441063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3c1f8-b563-45f0-967a-8f9820028a83 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:42.445206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 58/123] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:42 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:42.446022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3c1f8-b563-45f0-967a-8f9820028a83 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:42.505814 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 66/124] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:42.507865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f5676-3190-407b-bdbc-14bd4fd3fb1d None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:42.532835 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:42.633500 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3c1f8-b563-45f0-967a-8f9820028a83 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:42.702113 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f5676-3190-407b-bdbc-14bd4fd3fb1d None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:42.705223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb94278-b788-415c-b843-163805112fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:42.706850 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb94278-b788-415c-b843-163805112fec None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:42.707567 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb94278-b788-415c-b843-163805112fec None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:42.708838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb94278-b788-415c-b843-163805112fec None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:42.711518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb94278-b788-415c-b843-163805112fec None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:42.716364 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb94278-b788-415c-b843-163805112fec None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:42.718582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 59/125] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:42 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:42.719596 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb94278-b788-415c-b843-163805112fec None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:42.776872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9fe61-785b-47c5-b7e3-b04a5f6a7a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:42.777903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9fe61-785b-47c5-b7e3-b04a5f6a7a34 None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:42.778408 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9fe61-785b-47c5-b7e3-b04a5f6a7a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:42.778875 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9fe61-785b-47c5-b7e3-b04a5f6a7a34 None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3/users/1290a1aa40a149388a72943a7b2dc68e/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:42.782282 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9fe61-785b-47c5-b7e3-b04a5f6a7a34 None admin] RBAC: Authorizing `identity:create_grant(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3, user_id=1290a1aa40a149388a72943a7b2dc68e, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:42.786918 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9fe61-785b-47c5-b7e3-b04a5f6a7a34 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:42.810111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 67/126] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:08:42 2022] PUT /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3/users/1290a1aa40a149388a72943a7b2dc68e/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:08:42.817673 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9fe61-785b-47c5-b7e3-b04a5f6a7a34 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:42.828259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:42.929673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb94278-b788-415c-b843-163805112fec None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:42.965503 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:43.013495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c26ab-d4c7-49da-abe5-157254e903b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:43.014732 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c26ab-d4c7-49da-abe5-157254e903b9 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:43.015221 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c26ab-d4c7-49da-abe5-157254e903b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:43.016231 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c26ab-d4c7-49da-abe5-157254e903b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:43.029869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c26ab-d4c7-49da-abe5-157254e903b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:43.038150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-aa9c26ab-d4c7-49da-abe5-157254e903b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:08:43.038150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:43.038150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:43.038150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:43.038150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:43.038150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:43.038150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:43.038150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:43.038150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:43.038150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:43.038150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:43.038150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:43.038150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:43.038150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:43.038150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:43.038150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:43.039050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:43.039050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:43.039050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:43.039050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:43.039050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:43.039050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:43.039050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:43.039050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:43.039050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:43.039050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:43.039050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:43.039050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:43.039050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:43.039050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:43.039050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:43.039050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:43.039829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:43.039829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:43.039829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:43.039829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:43.039829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:43.039829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:43.039829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:43.039829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:43.039829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:43.039829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:43.039829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:08:43.039829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:08:43.040411 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9fe61-785b-47c5-b7e3-b04a5f6a7a34 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:43.042093 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 60/127] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:08:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:43.044041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c26ab-d4c7-49da-abe5-157254e903b9 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:43.088605 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024fe63-2493-4236-aac7-f4e38b401a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:43.089732 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024fe63-2493-4236-aac7-f4e38b401a8c None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:43.090184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024fe63-2493-4236-aac7-f4e38b401a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:43.090609 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024fe63-2493-4236-aac7-f4e38b401a8c None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3/users/a45fb43467b9498eb1b647735d0b88b5/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:43.098608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024fe63-2493-4236-aac7-f4e38b401a8c None admin] RBAC: Authorizing `identity:create_grant(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3, user_id=a45fb43467b9498eb1b647735d0b88b5, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:43.100457 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024fe63-2493-4236-aac7-f4e38b401a8c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:43.122164 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 68/128] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:08:42 2022] PUT /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3/users/a45fb43467b9498eb1b647735d0b88b5/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:08:43.123369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024fe63-2493-4236-aac7-f4e38b401a8c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:43.233651 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c26ab-d4c7-49da-abe5-157254e903b9 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:43.282459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:43.358926 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e092a-39c7-43c7-876d-1e6beb0d2711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:43.359589 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e092a-39c7-43c7-876d-1e6beb0d2711 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:43.359872 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e092a-39c7-43c7-876d-1e6beb0d2711 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:43.360151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e092a-39c7-43c7-876d-1e6beb0d2711 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:43.374589 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e092a-39c7-43c7-876d-1e6beb0d2711 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:43.380563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-6e2e092a-39c7-43c7-876d-1e6beb0d2711 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:08:43.380563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:43.380563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:43.380563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:43.380563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:43.380563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:43.380563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:43.380563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:43.380563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:43.380563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:43.380563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:43.380563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:43.380563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:43.380563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:43.380563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:43.380563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:43.381530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:43.381530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:43.381530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:43.381530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:43.381530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:43.381530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:43.381530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:43.381530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:43.381530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:43.381530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:43.381530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:43.381530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:43.381530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:43.381530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:43.381530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:43.381530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:43.382311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:43.382311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:43.382311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:43.382311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:43.382311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:43.382311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:43.382311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:43.382311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:43.382311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:43.382311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:43.382311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:08:43.382311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:08:43.382311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 61/129] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:43.387186 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024fe63-2493-4236-aac7-f4e38b401a8c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:43.389958 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e092a-39c7-43c7-876d-1e6beb0d2711 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:43.461560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:43.466466 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e288069-76e3-41b6-aeb4-0f515330bb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:43.467106 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e288069-76e3-41b6-aeb4-0f515330bb48 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:43.467431 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e288069-76e3-41b6-aeb4-0f515330bb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:43.467708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e288069-76e3-41b6-aeb4-0f515330bb48 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:43.468385 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e288069-76e3-41b6-aeb4-0f515330bb48 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:43.468691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e288069-76e3-41b6-aeb4-0f515330bb48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:43.470245 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e288069-76e3-41b6-aeb4-0f515330bb48 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:43.483951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 69/130] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:08:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:43.489990 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e288069-76e3-41b6-aeb4-0f515330bb48 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:43.540081 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e092a-39c7-43c7-876d-1e6beb0d2711 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:43.600028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cdae7-8a12-492e-9aa7-a4f2e0dfe1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:43.600651 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cdae7-8a12-492e-9aa7-a4f2e0dfe1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:43.600938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cdae7-8a12-492e-9aa7-a4f2e0dfe1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:43.601245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cdae7-8a12-492e-9aa7-a4f2e0dfe1f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:43.601955 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6cdae7-8a12-492e-9aa7-a4f2e0dfe1f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:43.602233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6cdae7-8a12-492e-9aa7-a4f2e0dfe1f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:43.603641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6cdae7-8a12-492e-9aa7-a4f2e0dfe1f2 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:43.619398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 62/131] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:08:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:43.620829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c7102-415f-4aee-ad5d-c6c909c9477c None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:43.621126 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c7102-415f-4aee-ad5d-c6c909c9477c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:43.621432 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c7102-415f-4aee-ad5d-c6c909c9477c None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:43.622646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 63/132] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:43.628671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c7102-415f-4aee-ad5d-c6c909c9477c None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:43.735490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e288069-76e3-41b6-aeb4-0f515330bb48 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:43.742444 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:43.751648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c7102-415f-4aee-ad5d-c6c909c9477c None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:43.784005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:43.790700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecad11f-9749-4dc9-b062-fe73d3181792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:43.791751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecad11f-9749-4dc9-b062-fe73d3181792 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:43.792208 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecad11f-9749-4dc9-b062-fe73d3181792 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:43.792647 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecad11f-9749-4dc9-b062-fe73d3181792 None admin] PATH_INFO: `/v3/users/a45fb43467b9498eb1b647735d0b88b5` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:43.794155 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecad11f-9749-4dc9-b062-fe73d3181792 None admin] RBAC: Authorizing `identity:get_user(user_id=a45fb43467b9498eb1b647735d0b88b5)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:43.795777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecad11f-9749-4dc9-b062-fe73d3181792 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:43.797882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 64/133] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:43 2022] GET /identity/v3/users/a45fb43467b9498eb1b647735d0b88b5 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:43.799283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712269e-9765-49eb-bee8-fa8732afb14d None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:43.800369 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712269e-9765-49eb-bee8-fa8732afb14d None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:43.800835 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712269e-9765-49eb-bee8-fa8732afb14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:43.802854 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0cfa7-2396-457f-8d69-444d0bb233f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:43.803521 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0cfa7-2396-457f-8d69-444d0bb233f6 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:43.803869 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0cfa7-2396-457f-8d69-444d0bb233f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:43.804154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0cfa7-2396-457f-8d69-444d0bb233f6 None admin] PATH_INFO: `/v3/users/a45fb43467b9498eb1b647735d0b88b5` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:43.811641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0cfa7-2396-457f-8d69-444d0bb233f6 None admin] RBAC: Authorizing `identity:get_user(user_id=a45fb43467b9498eb1b647735d0b88b5)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:43.811641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0cfa7-2396-457f-8d69-444d0bb233f6 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:43.811641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 70/134] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:43 2022] GET /identity/v3/users/a45fb43467b9498eb1b647735d0b88b5 => generated 275 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:43.812545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0cfa7-2396-457f-8d69-444d0bb233f6 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:43.835952 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-b712269e-9765-49eb-bee8-fa8732afb14d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:43.837512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b712269e-9765-49eb-bee8-fa8732afb14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:43.934433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 65/135] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:43.935313 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712269e-9765-49eb-bee8-fa8732afb14d None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:43.980565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0cfa7-2396-457f-8d69-444d0bb233f6 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:44.042732 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8f003-2a7d-47eb-8f63-94fdb305c6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:44.043297 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8f003-2a7d-47eb-8f63-94fdb305c6be None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:44.043586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8f003-2a7d-47eb-8f63-94fdb305c6be None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:44.043867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8f003-2a7d-47eb-8f63-94fdb305c6be None admin] PATH_INFO: `/v3/projects/dee48d9648274dd1a8f35ba629aa41b0` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:44.045806 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a8f003-2a7d-47eb-8f63-94fdb305c6be None admin] RBAC: Authorizing `identity:get_project(project_id=dee48d9648274dd1a8f35ba629aa41b0)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:44.047261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a8f003-2a7d-47eb-8f63-94fdb305c6be None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:44.048903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 71/136] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:43 2022] GET /identity/v3/projects/dee48d9648274dd1a8f35ba629aa41b0 => generated 312 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:44.056140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8f003-2a7d-47eb-8f63-94fdb305c6be None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:44.116262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:44.126340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712269e-9765-49eb-bee8-fa8732afb14d None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:44.187644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:44.201282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce83ce0-f9cb-4240-849c-f8b2b00ccf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:44.201935 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce83ce0-f9cb-4240-849c-f8b2b00ccf27 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:44.202218 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce83ce0-f9cb-4240-849c-f8b2b00ccf27 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:44.202497 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce83ce0-f9cb-4240-849c-f8b2b00ccf27 None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:44.205638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce83ce0-f9cb-4240-849c-f8b2b00ccf27 None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:44.212534 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce83ce0-f9cb-4240-849c-f8b2b00ccf27 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:44.216502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 66/137] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:43 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:44.219906 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce83ce0-f9cb-4240-849c-f8b2b00ccf27 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:44.261749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8f003-2a7d-47eb-8f63-94fdb305c6be None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:44.308517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cacdae-dd72-4746-9939-ef95c7fc3c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:44.309948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cacdae-dd72-4746-9939-ef95c7fc3c44 None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:44.310634 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cacdae-dd72-4746-9939-ef95c7fc3c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:44.311079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cacdae-dd72-4746-9939-ef95c7fc3c44 None admin] PATH_INFO: `/v3/projects/dee48d9648274dd1a8f35ba629aa41b0/users/a45fb43467b9498eb1b647735d0b88b5/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:44.313922 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cacdae-dd72-4746-9939-ef95c7fc3c44 None admin] RBAC: Authorizing `identity:create_grant(project_id=dee48d9648274dd1a8f35ba629aa41b0, user_id=a45fb43467b9498eb1b647735d0b88b5, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:44.316086 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cacdae-dd72-4746-9939-ef95c7fc3c44 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:44.343221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 72/138] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:08:44 2022] PUT /identity/v3/projects/dee48d9648274dd1a8f35ba629aa41b0/users/a45fb43467b9498eb1b647735d0b88b5/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:08:44.344880 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4769cec-06f2-4e48-b7f1-06af70b0544c None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:44.345332 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4769cec-06f2-4e48-b7f1-06af70b0544c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:44.345749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4769cec-06f2-4e48-b7f1-06af70b0544c None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:44.347230 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 73/139] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:44.358653 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617a96a-a2c3-4086-a395-2cef5a5a4221 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:44.359290 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617a96a-a2c3-4086-a395-2cef5a5a4221 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:44.359788 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617a96a-a2c3-4086-a395-2cef5a5a4221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:44.362189 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce83ce0-f9cb-4240-849c-f8b2b00ccf27 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:44.396171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e617a96a-a2c3-4086-a395-2cef5a5a4221 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:44.397782 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e617a96a-a2c3-4086-a395-2cef5a5a4221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:44.406181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e5ad0-1652-4527-b103-aa03531e3a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:44.407600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e5ad0-1652-4527-b103-aa03531e3a95 None admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:44.408109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e5ad0-1652-4527-b103-aa03531e3a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:44.408527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e5ad0-1652-4527-b103-aa03531e3a95 None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3/users/a45fb43467b9498eb1b647735d0b88b5/roles/8e9576d16f194938b86a1d1e087895bd` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:44.411494 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e5ad0-1652-4527-b103-aa03531e3a95 None admin] RBAC: Authorizing `identity:create_grant(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3, user_id=a45fb43467b9498eb1b647735d0b88b5, role_id=8e9576d16f194938b86a1d1e087895bd)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:44.413396 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e5ad0-1652-4527-b103-aa03531e3a95 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:44.436960 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 67/140] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:08:44 2022] PUT /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3/users/a45fb43467b9498eb1b647735d0b88b5/roles/8e9576d16f194938b86a1d1e087895bd => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:08:44.441910 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e5ad0-1652-4527-b103-aa03531e3a95 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:44.502290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:44.566478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 74/141] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:44.626719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e5ad0-1652-4527-b103-aa03531e3a95 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:44.669318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb470d-c56a-4825-8d80-6e3269926e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:44.670421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb470d-c56a-4825-8d80-6e3269926e30 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:44.670972 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb470d-c56a-4825-8d80-6e3269926e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:44.671455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb470d-c56a-4825-8d80-6e3269926e30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:44.672873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eb470d-c56a-4825-8d80-6e3269926e30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:44.675153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eb470d-c56a-4825-8d80-6e3269926e30 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:44.677502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 68/142] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:08:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:44.682985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617a96a-a2c3-4086-a395-2cef5a5a4221 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:44.762838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:44.814511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617a96a-a2c3-4086-a395-2cef5a5a4221 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:44.852511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94fd5b2-5bf8-42c9-8de3-19fd68347f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:44.853058 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fd5b2-5bf8-42c9-8de3-19fd68347f10 None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:44.853412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fd5b2-5bf8-42c9-8de3-19fd68347f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:44.853691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fd5b2-5bf8-42c9-8de3-19fd68347f10 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:44.854338 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94fd5b2-5bf8-42c9-8de3-19fd68347f10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:44.855782 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94fd5b2-5bf8-42c9-8de3-19fd68347f10 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:44.886666 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 75/143] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:08:44 2022] POST /identity/v3/users => generated 283 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:08:45.142757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb470d-c56a-4825-8d80-6e3269926e30 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:45.269416 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:45.283797 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb470d-c56a-4825-8d80-6e3269926e30 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:45.321459 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63bc04-afcf-4032-8acc-a9ba65c0a567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:45.322524 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63bc04-afcf-4032-8acc-a9ba65c0a567 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:45.323008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63bc04-afcf-4032-8acc-a9ba65c0a567 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:45.323492 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63bc04-afcf-4032-8acc-a9ba65c0a567 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:45.335418 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63bc04-afcf-4032-8acc-a9ba65c0a567 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:45.340681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-ef63bc04-afcf-4032-8acc-a9ba65c0a567 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:08:45.340681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:45.340681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:45.340681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:45.340681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:45.340681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:45.340681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:45.340681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:45.340681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:45.340681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:45.340681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:45.340681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:45.340681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:45.340681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:45.340681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:45.340681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:45.341538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:45.341538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:45.341538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:45.341538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:45.341538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:45.341538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:45.341538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:45.341538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:45.341538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:45.341538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:45.341538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:45.341538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:45.341538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:45.341538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:45.341538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:45.341538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:45.342322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:45.342322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:45.342322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:45.342322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:45.342322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:45.342322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:45.342322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:45.342322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:45.342322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:45.342322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:45.342322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:08:45.342322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:08:45.344881 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 69/144] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:45.354242 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94fd5b2-5bf8-42c9-8de3-19fd68347f10 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:45.494084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94fd5b2-5bf8-42c9-8de3-19fd68347f10 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:45.504594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:45.536889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b3220-0f1c-4e22-b88f-fff505201b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:45.537549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b3220-0f1c-4e22-b88f-fff505201b45 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:45.537842 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b3220-0f1c-4e22-b88f-fff505201b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:45.538120 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b3220-0f1c-4e22-b88f-fff505201b45 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:45.538807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b3220-0f1c-4e22-b88f-fff505201b45 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:45.539117 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b3220-0f1c-4e22-b88f-fff505201b45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:45.540514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b3220-0f1c-4e22-b88f-fff505201b45 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:45.553473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 76/145] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:08:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:45.559333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63bc04-afcf-4032-8acc-a9ba65c0a567 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:45.704118 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63bc04-afcf-4032-8acc-a9ba65c0a567 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:45.746199 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1cc41-8e0c-4338-b47e-07252c7e7a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:45.747153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1cc41-8e0c-4338-b47e-07252c7e7a4a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:45.747632 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1cc41-8e0c-4338-b47e-07252c7e7a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:45.748115 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1cc41-8e0c-4338-b47e-07252c7e7a4a None admin] PATH_INFO: `/v3/users/17f1c3d7516145758cee5edf5ebc5888` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:45.749498 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1cc41-8e0c-4338-b47e-07252c7e7a4a None admin] RBAC: Authorizing `identity:get_user(user_id=17f1c3d7516145758cee5edf5ebc5888)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:45.751132 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1cc41-8e0c-4338-b47e-07252c7e7a4a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:45.753815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 70/146] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:45 2022] GET /identity/v3/users/17f1c3d7516145758cee5edf5ebc5888 => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:45.759797 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b3220-0f1c-4e22-b88f-fff505201b45 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:45.891584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b3220-0f1c-4e22-b88f-fff505201b45 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:45.929737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00913469-fcb0-43a9-a690-b943f11cca4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:45.930397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00913469-fcb0-43a9-a690-b943f11cca4a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:45.930681 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00913469-fcb0-43a9-a690-b943f11cca4a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:45.930961 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00913469-fcb0-43a9-a690-b943f11cca4a None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:45.932124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00913469-fcb0-43a9-a690-b943f11cca4a None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:45.933616 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00913469-fcb0-43a9-a690-b943f11cca4a None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:45.935279 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 77/147] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:45 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:45.942389 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1cc41-8e0c-4338-b47e-07252c7e7a4a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:45.974315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:46.012778 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19a467-cb54-47e2-bf4c-517869ddda95 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:46.013068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19a467-cb54-47e2-bf4c-517869ddda95 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:46.013331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19a467-cb54-47e2-bf4c-517869ddda95 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:46.014717 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 78/148] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:46.021071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1eb5e-66b3-4d9d-bf3d-142462f9e323 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:46.025165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1eb5e-66b3-4d9d-bf3d-142462f9e323 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:46.025165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1eb5e-66b3-4d9d-bf3d-142462f9e323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:46.055653 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-3ad1eb5e-66b3-4d9d-bf3d-142462f9e323 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:46.056862 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-3ad1eb5e-66b3-4d9d-bf3d-142462f9e323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:46.126599 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:46.132794 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1cc41-8e0c-4338-b47e-07252c7e7a4a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:46.200776 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 79/149] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:08:46.214686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ac813-6106-43d4-9cf9-06a425d1ff65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:46.215632 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ac813-6106-43d4-9cf9-06a425d1ff65 None admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:46.216109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ac813-6106-43d4-9cf9-06a425d1ff65 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:46.216636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ac813-6106-43d4-9cf9-06a425d1ff65 None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3/users/17f1c3d7516145758cee5edf5ebc5888/roles/edf27bedabd24681b24ec2a72a774a9c` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:46.221436 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ac813-6106-43d4-9cf9-06a425d1ff65 None admin] RBAC: Authorizing `identity:create_grant(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3, user_id=17f1c3d7516145758cee5edf5ebc5888, role_id=edf27bedabd24681b24ec2a72a774a9c)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:46.223397 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ac813-6106-43d4-9cf9-06a425d1ff65 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:46.249459 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 71/150] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:08:45 2022] PUT /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3/users/17f1c3d7516145758cee5edf5ebc5888/roles/edf27bedabd24681b24ec2a72a774a9c => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:08:46.494511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150fcb9-e938-452b-806e-0a77481c125a None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:46.494933 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150fcb9-e938-452b-806e-0a77481c125a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:46.495107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150fcb9-e938-452b-806e-0a77481c125a None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:46.496594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 80/151] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:46.502760 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad875fa-cb64-417c-9496-fa4e2e7b1db0 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:46.503613 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad875fa-cb64-417c-9496-fa4e2e7b1db0 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:46.504119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad875fa-cb64-417c-9496-fa4e2e7b1db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:46.528765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-4ad875fa-cb64-417c-9496-fa4e2e7b1db0 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:46.530377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-4ad875fa-cb64-417c-9496-fa4e2e7b1db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:46.613377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 72/152] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:46.869913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150fcb9-e938-452b-806e-0a77481c125a None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:46.984960 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150fcb9-e938-452b-806e-0a77481c125a None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:47.024473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113eef8b-3b76-45ad-b250-867539df39fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:47.025119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113eef8b-3b76-45ad-b250-867539df39fb None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:47.025423 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113eef8b-3b76-45ad-b250-867539df39fb None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:47.025699 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113eef8b-3b76-45ad-b250-867539df39fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:47.042268 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113eef8b-3b76-45ad-b250-867539df39fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:47.047184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-113eef8b-3b76-45ad-b250-867539df39fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:08:47.047184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:47.047184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:47.047184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:47.047184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:47.047184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:47.047184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:47.047184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:47.047184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:47.047184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:47.047184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:47.047184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:47.047184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:47.047184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:47.047184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:47.047184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:47.048114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:47.048114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:47.048114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:47.048114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:47.048114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:47.048114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:47.048114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:47.048114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:47.048114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:47.048114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:47.048114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:47.048114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:47.048114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:47.048114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:47.048114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:47.048114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:47.048865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:47.048865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:47.048865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:47.048865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:47.048865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:47.048865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:47.048865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:47.048865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:47.048865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:47.048865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:47.048865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:08:47.048865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:08:47.048865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 81/153] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:08:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:47.054872 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad875fa-cb64-417c-9496-fa4e2e7b1db0 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:47.070542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:47.200449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad875fa-cb64-417c-9496-fa4e2e7b1db0 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:47.210183 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113eef8b-3b76-45ad-b250-867539df39fb None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:47.255824 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846c5cd-debb-4bce-9808-bc6672015f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:47.256442 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846c5cd-debb-4bce-9808-bc6672015f27 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:47.256716 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846c5cd-debb-4bce-9808-bc6672015f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:47.256980 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846c5cd-debb-4bce-9808-bc6672015f27 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:47.257669 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846c5cd-debb-4bce-9808-bc6672015f27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:47.257967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846c5cd-debb-4bce-9808-bc6672015f27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:47.259934 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846c5cd-debb-4bce-9808-bc6672015f27 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:47.269706 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 73/154] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:47.276404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846c5cd-debb-4bce-9808-bc6672015f27 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:47.277683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:47.380068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:47.413997 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113eef8b-3b76-45ad-b250-867539df39fb None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:47.420296 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846c5cd-debb-4bce-9808-bc6672015f27 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:47.458864 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbd7ef-b8ab-4b20-8df2-26c8b9aa1893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:47.459507 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbd7ef-b8ab-4b20-8df2-26c8b9aa1893 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:47.459784 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbd7ef-b8ab-4b20-8df2-26c8b9aa1893 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:47.460055 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbd7ef-b8ab-4b20-8df2-26c8b9aa1893 None admin] PATH_INFO: `/v3/users/1290a1aa40a149388a72943a7b2dc68e` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:47.461896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbd7ef-b8ab-4b20-8df2-26c8b9aa1893 None admin] RBAC: Authorizing `identity:get_user(user_id=1290a1aa40a149388a72943a7b2dc68e)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:47.463333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbd7ef-b8ab-4b20-8df2-26c8b9aa1893 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:47.464920 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 74/155] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:47 2022] GET /identity/v3/users/1290a1aa40a149388a72943a7b2dc68e => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:47.474179 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbd7ef-b8ab-4b20-8df2-26c8b9aa1893 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:47.482346 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37f32e-c382-46d8-a198-0c2355b2ae2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:47.482987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37f32e-c382-46d8-a198-0c2355b2ae2c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:47.483267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37f32e-c382-46d8-a198-0c2355b2ae2c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:47.483538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37f32e-c382-46d8-a198-0c2355b2ae2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:47.513677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37f32e-c382-46d8-a198-0c2355b2ae2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:47.519072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-8a37f32e-c382-46d8-a198-0c2355b2ae2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:08:47.519072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:47.519072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:47.519072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:47.519072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:47.519072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:47.519072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:47.519072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:47.519072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:47.519072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:47.519072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:47.519072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:47.519072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:47.519072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:47.519072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:47.519072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:47.520192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:47.520192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:47.520192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:47.520192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:47.520192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:47.520192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:47.520192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:47.520192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:47.520192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:47.520192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:47.520192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:47.520192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:47.520192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:47.520192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:47.520192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:47.520192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:47.521042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:47.521042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:47.521042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:47.521042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:47.521042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:47.521042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:47.521042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:47.521042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:47.521042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:47.521042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:47.521042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:08:47.521042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:08:47.528226 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 82/156] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:47.552886 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37f32e-c382-46d8-a198-0c2355b2ae2c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:47.675530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbd7ef-b8ab-4b20-8df2-26c8b9aa1893 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:47.714366 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:47.741763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37f32e-c382-46d8-a198-0c2355b2ae2c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:47.769646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd7c4c-409e-4e8b-b182-30de7401741a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:47.770267 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd7c4c-409e-4e8b-b182-30de7401741a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:47.770543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd7c4c-409e-4e8b-b182-30de7401741a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:47.770810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bd7c4c-409e-4e8b-b182-30de7401741a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:47.783560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:47.805853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800283f-c88a-498b-89ce-daa251a8b1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:47.806860 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800283f-c88a-498b-89ce-daa251a8b1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:47.807396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800283f-c88a-498b-89ce-daa251a8b1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:47.807879 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800283f-c88a-498b-89ce-daa251a8b1f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:47.808829 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800283f-c88a-498b-89ce-daa251a8b1f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:47.809368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800283f-c88a-498b-89ce-daa251a8b1f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:47.811034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800283f-c88a-498b-89ce-daa251a8b1f6 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:47.821182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 83/157] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:08:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:47.829546 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800283f-c88a-498b-89ce-daa251a8b1f6 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:47.920525 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd7c4c-409e-4e8b-b182-30de7401741a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:47.922025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bd7c4c-409e-4e8b-b182-30de7401741a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:48.016698 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800283f-c88a-498b-89ce-daa251a8b1f6 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:48.090905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e94a54-22d7-405c-a846-52c8dc758d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:48.091566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e94a54-22d7-405c-a846-52c8dc758d0b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:48.091847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e94a54-22d7-405c-a846-52c8dc758d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:48.092119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e94a54-22d7-405c-a846-52c8dc758d0b None admin] PATH_INFO: `/v3/users/a45fb43467b9498eb1b647735d0b88b5` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:48.097965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e94a54-22d7-405c-a846-52c8dc758d0b None admin] RBAC: Authorizing `identity:get_user(user_id=a45fb43467b9498eb1b647735d0b88b5)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:48.099471 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e94a54-22d7-405c-a846-52c8dc758d0b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:48.101116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 84/158] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:47 2022] GET /identity/v3/users/a45fb43467b9498eb1b647735d0b88b5 => generated 275 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:48.109579 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e94a54-22d7-405c-a846-52c8dc758d0b None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:48.153789 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 75/159] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:08:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 680 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:08:48.155933 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:48.161882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd7c4c-409e-4e8b-b182-30de7401741a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:48.233530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:48.325305 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bd7c4c-409e-4e8b-b182-30de7401741a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:48.348190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e94a54-22d7-405c-a846-52c8dc758d0b None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:48.385856 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8091fc-2bc9-4b73-8331-804adf2a89ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:48.386508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8091fc-2bc9-4b73-8331-804adf2a89ad None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:48.386782 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8091fc-2bc9-4b73-8331-804adf2a89ad None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:48.387052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8091fc-2bc9-4b73-8331-804adf2a89ad None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:48.388213 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8091fc-2bc9-4b73-8331-804adf2a89ad None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:48.389678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8091fc-2bc9-4b73-8331-804adf2a89ad None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:48.391314 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 76/160] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:48 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:48.392434 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a8ce8-fcef-49eb-bb2a-8139de4ef7e0 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:48.392709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a8ce8-fcef-49eb-bb2a-8139de4ef7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:48.392977 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a8ce8-fcef-49eb-bb2a-8139de4ef7e0 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:48.394964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 77/161] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:48.400358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c1572-0f49-409b-878b-2dc2a22c3087 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:48.400640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c1572-0f49-409b-878b-2dc2a22c3087 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:48.400913 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c1572-0f49-409b-878b-2dc2a22c3087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:48.429869 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493256f-517a-43df-a7c6-7e1de7427926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:48.430121 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-5b1c1572-0f49-409b-878b-2dc2a22c3087 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:48.430501 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493256f-517a-43df-a7c6-7e1de7427926 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:48.430786 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493256f-517a-43df-a7c6-7e1de7427926 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:48.431064 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493256f-517a-43df-a7c6-7e1de7427926 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:48.431291 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5b1c1572-0f49-409b-878b-2dc2a22c3087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:48.533319 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 78/162] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:48.534440 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c1572-0f49-409b-878b-2dc2a22c3087 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:48.594242 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493256f-517a-43df-a7c6-7e1de7427926 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:48.595970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:48.596099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493256f-517a-43df-a7c6-7e1de7427926 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:48.645416 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:48.689460 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c1572-0f49-409b-878b-2dc2a22c3087 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:48.745249 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b3993-7913-4df7-9ae5-fd20e8b2a98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:48.745864 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b3993-7913-4df7-9ae5-fd20e8b2a98c None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:48.746140 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b3993-7913-4df7-9ae5-fd20e8b2a98c None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:48.746408 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b3993-7913-4df7-9ae5-fd20e8b2a98c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:48.747128 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7b3993-7913-4df7-9ae5-fd20e8b2a98c None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3, user.id=1290a1aa40a149388a72943a7b2dc68e) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:48.747431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7b3993-7913-4df7-9ae5-fd20e8b2a98c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:48.748866 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7b3993-7913-4df7-9ae5-fd20e8b2a98c None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:48.757259 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 79/163] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:08:48 2022] GET /identity/v3/role_assignments?user.id=1290a1aa40a149388a72943a7b2dc68e&scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:48.758454 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760ef93-69a3-4f0f-ab47-84c43e53b07a None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:48.758734 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760ef93-69a3-4f0f-ab47-84c43e53b07a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:48.759008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760ef93-69a3-4f0f-ab47-84c43e53b07a None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:48.760200 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 80/164] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:48.766111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb037ffd-18c9-438c-bcd6-f704d8599380 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:48.766429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb037ffd-18c9-438c-bcd6-f704d8599380 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:48.766657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb037ffd-18c9-438c-bcd6-f704d8599380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:48.792768 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-eb037ffd-18c9-438c-bcd6-f704d8599380 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:48.793999 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-eb037ffd-18c9-438c-bcd6-f704d8599380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:48.824571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 85/165] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:08:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 715 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 12:08:48.837520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493256f-517a-43df-a7c6-7e1de7427926 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:48.899772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 81/166] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:08:49.019958 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493256f-517a-43df-a7c6-7e1de7427926 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:49.046001 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:49.072610 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4fe98-38e7-42d6-a499-8cddb94e4f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:49.073286 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4fe98-38e7-42d6-a499-8cddb94e4f15 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:49.073570 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4fe98-38e7-42d6-a499-8cddb94e4f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:49.073845 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4fe98-38e7-42d6-a499-8cddb94e4f15 None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:49.074985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4fe98-38e7-42d6-a499-8cddb94e4f15 None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:49.076442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4fe98-38e7-42d6-a499-8cddb94e4f15 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:49.079074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 86/167] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:48 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:49.084519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb037ffd-18c9-438c-bcd6-f704d8599380 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:49.141555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:49.231916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb037ffd-18c9-438c-bcd6-f704d8599380 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:49.279126 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abac231-8534-47c6-a138-85e641936162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:49.279764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abac231-8534-47c6-a138-85e641936162 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:49.280033 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abac231-8534-47c6-a138-85e641936162 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:49.280306 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abac231-8534-47c6-a138-85e641936162 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:49.281044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abac231-8534-47c6-a138-85e641936162 None admin] RBAC: Adding query filter params (role.id=2201e005d5ef449896a0e68a940678ec, scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3, user.id=a45fb43467b9498eb1b647735d0b88b5) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:49.281377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abac231-8534-47c6-a138-85e641936162 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:49.282828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abac231-8534-47c6-a138-85e641936162 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:49.289478 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 82/168] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:08:49 2022] GET /identity/v3/role_assignments?user.id=a45fb43467b9498eb1b647735d0b88b5&scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3&role.id=2201e005d5ef449896a0e68a940678ec => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:49.328065 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4fe98-38e7-42d6-a499-8cddb94e4f15 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:49.427685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abac231-8534-47c6-a138-85e641936162 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:49.464560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4fe98-38e7-42d6-a499-8cddb94e4f15 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:49.541474 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7d2a8-9108-405a-bbb9-2ce5eef9da03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:49.542190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7d2a8-9108-405a-bbb9-2ce5eef9da03 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:49.542479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7d2a8-9108-405a-bbb9-2ce5eef9da03 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:49.542809 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7d2a8-9108-405a-bbb9-2ce5eef9da03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:49.570634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:49.573846 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7d2a8-9108-405a-bbb9-2ce5eef9da03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:49.579028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-29a7d2a8-9108-405a-bbb9-2ce5eef9da03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:08:49.579028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:49.579028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:49.579028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:49.579028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:49.579028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:49.579028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:49.579028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:49.579028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:49.579028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:49.579028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:49.579028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:49.579028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:49.579028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:49.579028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:49.579028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:49.579956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:49.579956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:49.579956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:49.579956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:49.579956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:49.579956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:49.579956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:49.579956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:49.579956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:49.579956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:49.579956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:49.579956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:49.579956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:49.579956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:49.579956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:49.579956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:49.581462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:49.581462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:49.581462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:49.581462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:49.581462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:49.581462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:49.581462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:49.581462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:49.581462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:49.581462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:49.581462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:08:49.581462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:08:49.581462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 87/169] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:49.582165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d22bda-f765-4703-9759-02fe1291e7ca None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:49.582236 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d22bda-f765-4703-9759-02fe1291e7ca None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:49.582824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d22bda-f765-4703-9759-02fe1291e7ca None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:49.582919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abac231-8534-47c6-a138-85e641936162 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:49.583753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 88/170] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:49.589740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f4078-f6da-4e63-8d69-0ef5fa662d41 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:49.590267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f4078-f6da-4e63-8d69-0ef5fa662d41 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:49.590693 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f4078-f6da-4e63-8d69-0ef5fa662d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:49.630166 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f86e3b-afc1-4d36-8dbe-e21d9e1abd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:49.630561 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-494f4078-f6da-4e63-8d69-0ef5fa662d41 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:49.631063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f86e3b-afc1-4d36-8dbe-e21d9e1abd0e None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:49.631438 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f86e3b-afc1-4d36-8dbe-e21d9e1abd0e None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:49.631803 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f86e3b-afc1-4d36-8dbe-e21d9e1abd0e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:49.632505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-494f4078-f6da-4e63-8d69-0ef5fa662d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:49.660609 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f86e3b-afc1-4d36-8dbe-e21d9e1abd0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:49.666783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-e1f86e3b-afc1-4d36-8dbe-e21d9e1abd0e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:08:49.666783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:49.666783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:49.666783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:49.666783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:49.666783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:49.666783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:49.666783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:49.666783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:49.666783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:49.666783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:49.666783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:49.666783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:49.666783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:49.666783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:49.666783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:49.667736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:49.667736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:49.667736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:49.667736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:49.667736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:49.667736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:49.667736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:49.667736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:49.667736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:49.667736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:49.667736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:49.667736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:49.667736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:49.667736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:49.667736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:49.667736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:49.668607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:49.668607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:49.668607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:49.668607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:49.668607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:49.668607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:49.668607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:49.668607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:49.668607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:49.668607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:49.668607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:08:49.668607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:08:49.668607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 83/171] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:08:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:08:49.669302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f86e3b-afc1-4d36-8dbe-e21d9e1abd0e None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:49.747680 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:49.819694 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 89/172] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:49.820406 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f4078-f6da-4e63-8d69-0ef5fa662d41 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:49.820717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f86e3b-afc1-4d36-8dbe-e21d9e1abd0e None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:49.881851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cdd4a-8b33-4164-8b6d-1ad8dc38baf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:49.882518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cdd4a-8b33-4164-8b6d-1ad8dc38baf7 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:49.882813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cdd4a-8b33-4164-8b6d-1ad8dc38baf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:49.883067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375cdd4a-8b33-4164-8b6d-1ad8dc38baf7 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:49.883789 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cdd4a-8b33-4164-8b6d-1ad8dc38baf7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:49.884093 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cdd4a-8b33-4164-8b6d-1ad8dc38baf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:49.887850 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375cdd4a-8b33-4164-8b6d-1ad8dc38baf7 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:49.898876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 84/173] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:08:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:49.904818 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cdd4a-8b33-4164-8b6d-1ad8dc38baf7 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:49.927447 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:50.050280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375cdd4a-8b33-4164-8b6d-1ad8dc38baf7 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:50.095230 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f4078-f6da-4e63-8d69-0ef5fa662d41 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:50.105856 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4062a7d-4ac4-49a1-af04-61d5df5df85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:50.106501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4062a7d-4ac4-49a1-af04-61d5df5df85b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:50.107039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4062a7d-4ac4-49a1-af04-61d5df5df85b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:50.107039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4062a7d-4ac4-49a1-af04-61d5df5df85b None admin] PATH_INFO: `/v3/users/a45fb43467b9498eb1b647735d0b88b5` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:50.108213 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4062a7d-4ac4-49a1-af04-61d5df5df85b None admin] RBAC: Authorizing `identity:get_user(user_id=a45fb43467b9498eb1b647735d0b88b5)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:50.109691 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4062a7d-4ac4-49a1-af04-61d5df5df85b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:50.111768 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 85/174] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:49 2022] GET /identity/v3/users/a45fb43467b9498eb1b647735d0b88b5 => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:50.118520 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4062a7d-4ac4-49a1-af04-61d5df5df85b None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:50.190530 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e261f-9721-47a4-b6d7-f5afe4e506e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:50.191167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e261f-9721-47a4-b6d7-f5afe4e506e0 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:50.191490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e261f-9721-47a4-b6d7-f5afe4e506e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:50.191716 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e261f-9721-47a4-b6d7-f5afe4e506e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:50.192390 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e261f-9721-47a4-b6d7-f5afe4e506e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:50.192691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e261f-9721-47a4-b6d7-f5afe4e506e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:50.194112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e261f-9721-47a4-b6d7-f5afe4e506e0 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:50.212177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 90/175] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:08:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:50.217686 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e261f-9721-47a4-b6d7-f5afe4e506e0 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:50.279724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:50.304445 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4062a7d-4ac4-49a1-af04-61d5df5df85b None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:50.323374 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:50.376331 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0437c11-aa0a-442c-b277-731e8808339e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:50.377372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0437c11-aa0a-442c-b277-731e8808339e None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:50.378425 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0437c11-aa0a-442c-b277-731e8808339e None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:50.379219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0437c11-aa0a-442c-b277-731e8808339e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:50.459309 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e261f-9721-47a4-b6d7-f5afe4e506e0 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:50.492665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0437c11-aa0a-442c-b277-731e8808339e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:50.495304 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0437c11-aa0a-442c-b277-731e8808339e None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:50.499192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03bd24-f9f0-4a3e-8b98-21a2492cbc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:50.499840 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03bd24-f9f0-4a3e-8b98-21a2492cbc6a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:50.500125 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03bd24-f9f0-4a3e-8b98-21a2492cbc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:50.500403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03bd24-f9f0-4a3e-8b98-21a2492cbc6a None admin] PATH_INFO: `/v3/users/a45fb43467b9498eb1b647735d0b88b5` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:50.501587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03bd24-f9f0-4a3e-8b98-21a2492cbc6a None admin] RBAC: Authorizing `identity:get_user(user_id=a45fb43467b9498eb1b647735d0b88b5)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:50.503127 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03bd24-f9f0-4a3e-8b98-21a2492cbc6a None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:50.504724 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 91/176] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:50 2022] GET /identity/v3/users/a45fb43467b9498eb1b647735d0b88b5 => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:50.505942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee381d-146c-4f0a-917f-6fdca3c2f7e0 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:50.506233 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee381d-146c-4f0a-917f-6fdca3c2f7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:50.506505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee381d-146c-4f0a-917f-6fdca3c2f7e0 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:50.509429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 92/177] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:50.510217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee381d-146c-4f0a-917f-6fdca3c2f7e0 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:50.590450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:50.614749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 86/178] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:08:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 12:08:50.614749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bdb32-8c3e-45f3-b564-2a07f0ec346a None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:50.614749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bdb32-8c3e-45f3-b564-2a07f0ec346a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:50.614749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bdb32-8c3e-45f3-b564-2a07f0ec346a None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:50.614749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 87/179] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:50.614749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6bdb32-8c3e-45f3-b564-2a07f0ec346a None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:50.629826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee381d-146c-4f0a-917f-6fdca3c2f7e0 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:50.637676 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:50.669950 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c011a7-8633-4aec-8b4e-50c33551ddeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:50.670584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c011a7-8633-4aec-8b4e-50c33551ddeb None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:50.670868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c011a7-8633-4aec-8b4e-50c33551ddeb None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:50.671145 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c011a7-8633-4aec-8b4e-50c33551ddeb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:50.681645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c011a7-8633-4aec-8b4e-50c33551ddeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:50.686487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-e2c011a7-8633-4aec-8b4e-50c33551ddeb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:08:50.686487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:50.686487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:50.686487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:50.686487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:50.686487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:50.686487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:50.686487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:50.686487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:50.686487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:50.686487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:50.686487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:50.686487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:50.686487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:50.686487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:50.686487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:50.687377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:50.687377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:50.687377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:50.687377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:50.687377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:50.687377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:50.687377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:50.687377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:50.687377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:50.687377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:50.687377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:50.687377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:50.687377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:50.687377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:50.687377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:50.687377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:50.688160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:50.688160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:50.688160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:50.688160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:50.688160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:50.688160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:50.688160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:50.688160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:50.688160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:50.688160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:50.688160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:08:50.688160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:08:50.688160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 93/180] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:08:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:50.694124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d6459-6ca7-4a8a-96d0-52374becda32 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:50.694409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d6459-6ca7-4a8a-96d0-52374becda32 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:50.694684 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92d6459-6ca7-4a8a-96d0-52374becda32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:50.711878 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-c92d6459-6ca7-4a8a-96d0-52374becda32 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:50.713103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c92d6459-6ca7-4a8a-96d0-52374becda32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:50.718479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6bdb32-8c3e-45f3-b564-2a07f0ec346a None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:50.752274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a58384-4e62-4438-af45-c114a64c7557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:50.753546 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a58384-4e62-4438-af45-c114a64c7557 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:50.753837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a58384-4e62-4438-af45-c114a64c7557 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:50.754108 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a58384-4e62-4438-af45-c114a64c7557 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:50.778806 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 94/181] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:50.779633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d6459-6ca7-4a8a-96d0-52374becda32 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:50.807963 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:50.826272 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a58384-4e62-4438-af45-c114a64c7557 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:50.827735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a58384-4e62-4438-af45-c114a64c7557 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:50.888165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:50.894933 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92d6459-6ca7-4a8a-96d0-52374becda32 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:50.928510 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b765191-7c13-40cf-b3be-b7593ae55922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:50.929157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b765191-7c13-40cf-b3be-b7593ae55922 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:50.929461 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b765191-7c13-40cf-b3be-b7593ae55922 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:50.929736 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b765191-7c13-40cf-b3be-b7593ae55922 None admin] PATH_INFO: `/v3/projects/dee48d9648274dd1a8f35ba629aa41b0` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:50.930847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b765191-7c13-40cf-b3be-b7593ae55922 None admin] RBAC: Authorizing `identity:get_project(project_id=dee48d9648274dd1a8f35ba629aa41b0)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:50.932305 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b765191-7c13-40cf-b3be-b7593ae55922 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:50.938824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 95/182] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:50 2022] GET /identity/v3/projects/dee48d9648274dd1a8f35ba629aa41b0 => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:50.938824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254def6e-8539-40f9-a900-631bc545cea9 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:50.938824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254def6e-8539-40f9-a900-631bc545cea9 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:50.938824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254def6e-8539-40f9-a900-631bc545cea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:50.945115 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 88/183] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:08:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:08:50.946446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3d6d7-03a7-44b0-9371-92002a79a8b2 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:50.946732 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3d6d7-03a7-44b0-9371-92002a79a8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:50.947004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b3d6d7-03a7-44b0-9371-92002a79a8b2 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:50.948231 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 89/184] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:50.948923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3d6d7-03a7-44b0-9371-92002a79a8b2 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:50.959878 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-254def6e-8539-40f9-a900-631bc545cea9 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:50.961068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-254def6e-8539-40f9-a900-631bc545cea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:51.034305 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 96/185] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:51.035450 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254def6e-8539-40f9-a900-631bc545cea9 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:51.039620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:51.059647 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b3d6d7-03a7-44b0-9371-92002a79a8b2 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:51.095883 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb65e2-3175-4e25-9da2-a8d41eadf5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:51.097297 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb65e2-3175-4e25-9da2-a8d41eadf5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:51.097587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb65e2-3175-4e25-9da2-a8d41eadf5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:51.098163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb65e2-3175-4e25-9da2-a8d41eadf5f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:51.099188 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb65e2-3175-4e25-9da2-a8d41eadf5f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:51.099579 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb65e2-3175-4e25-9da2-a8d41eadf5f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:51.100894 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb65e2-3175-4e25-9da2-a8d41eadf5f2 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:51.113352 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 90/186] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:51.114329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb65e2-3175-4e25-9da2-a8d41eadf5f2 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:51.135757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:51.143751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254def6e-8539-40f9-a900-631bc545cea9 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:51.183624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2d330-2f3e-4016-b699-7259864066ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:51.184463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2d330-2f3e-4016-b699-7259864066ce None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:51.185151 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2d330-2f3e-4016-b699-7259864066ce None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:51.185489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2d330-2f3e-4016-b699-7259864066ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:51.186231 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2d330-2f3e-4016-b699-7259864066ce None admin] RBAC: Adding query filter params (role.id=2201e005d5ef449896a0e68a940678ec, scope.project.id=dee48d9648274dd1a8f35ba629aa41b0, user.id=a45fb43467b9498eb1b647735d0b88b5) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:51.186538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2d330-2f3e-4016-b699-7259864066ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:51.187994 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2d330-2f3e-4016-b699-7259864066ce None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:51.197182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 97/187] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:08:51 2022] GET /identity/v3/role_assignments?user.id=a45fb43467b9498eb1b647735d0b88b5&scope.project.id=dee48d9648274dd1a8f35ba629aa41b0&role.id=2201e005d5ef449896a0e68a940678ec => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:51.197182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee38a8f-9f46-4a6c-a920-8e63c009e896 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:51.197182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee38a8f-9f46-4a6c-a920-8e63c009e896 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:51.197182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee38a8f-9f46-4a6c-a920-8e63c009e896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:51.214556 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-6ee38a8f-9f46-4a6c-a920-8e63c009e896 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:51.215836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-6ee38a8f-9f46-4a6c-a920-8e63c009e896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:51.245291 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb65e2-3175-4e25-9da2-a8d41eadf5f2 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:51.288497 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd4575e-aca0-4602-bf17-e5b65247f409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:51.289020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4575e-aca0-4602-bf17-e5b65247f409 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:51.289332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4575e-aca0-4602-bf17-e5b65247f409 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:51.289604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd4575e-aca0-4602-bf17-e5b65247f409 None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:51.290917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd4575e-aca0-4602-bf17-e5b65247f409 None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:51.292341 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd4575e-aca0-4602-bf17-e5b65247f409 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:51.294187 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 91/188] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:51 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:51.300962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd4575e-aca0-4602-bf17-e5b65247f409 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:51.307677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 98/189] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:51.308609 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee38a8f-9f46-4a6c-a920-8e63c009e896 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:51.321419 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:51.412665 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:51.431091 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee38a8f-9f46-4a6c-a920-8e63c009e896 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:51.445304 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd4575e-aca0-4602-bf17-e5b65247f409 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:51.479098 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baee4ab-666d-45a5-ab71-5d6a30bfd7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:51.479888 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baee4ab-666d-45a5-ab71-5d6a30bfd7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:51.480020 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baee4ab-666d-45a5-ab71-5d6a30bfd7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:51.480321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baee4ab-666d-45a5-ab71-5d6a30bfd7a6 None admin] PATH_INFO: `/v3/users/d00c15ca137346aea94f5698900fe15c` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:51.491452 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e670e28-4e51-4936-bc6b-a2755b523eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:51.492077 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e670e28-4e51-4936-bc6b-a2755b523eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:51.492359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e670e28-4e51-4936-bc6b-a2755b523eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:51.492617 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e670e28-4e51-4936-bc6b-a2755b523eb3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:51.495554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baee4ab-666d-45a5-ab71-5d6a30bfd7a6 None admin] RBAC: Authorizing `identity:get_user(user_id=d00c15ca137346aea94f5698900fe15c)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:51.496999 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baee4ab-666d-45a5-ab71-5d6a30bfd7a6 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:51.498729 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 99/190] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:51 2022] GET /identity/v3/users/d00c15ca137346aea94f5698900fe15c => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:51.504089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baee4ab-666d-45a5-ab71-5d6a30bfd7a6 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:51.508883 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e670e28-4e51-4936-bc6b-a2755b523eb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:51.514351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-7e670e28-4e51-4936-bc6b-a2755b523eb3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:08:51.514351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:51.514351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:51.514351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:51.514351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:51.514351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:51.514351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:51.514351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:51.514351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:51.514351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:51.514351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:51.514351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:51.514351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:51.514351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:51.514351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:51.514351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:51.515225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:51.515225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:51.515225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:51.515225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:51.515225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:51.515225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:51.515225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:51.515225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:51.515225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:51.515225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:51.515225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:51.515225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:51.515225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:51.515225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:51.515225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:51.515225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:51.516111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:51.516111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:51.516111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:51.516111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:51.516111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:51.516111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:51.516111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:51.516111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:51.516111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:51.516111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:51.516111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:08:51.516111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:08:51.516111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 92/191] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:51.516785 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e670e28-4e51-4936-bc6b-a2755b523eb3 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:51.591069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:51.611289 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baee4ab-666d-45a5-ab71-5d6a30bfd7a6 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:51.616262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e670e28-4e51-4936-bc6b-a2755b523eb3 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:51.645198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7f5ec-173c-4c83-9513-779866c093b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:51.645807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7f5ec-173c-4c83-9513-779866c093b6 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:51.646088 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7f5ec-173c-4c83-9513-779866c093b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:51.646364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7f5ec-173c-4c83-9513-779866c093b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:51.647084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7f5ec-173c-4c83-9513-779866c093b6 None admin] RBAC: Adding query filter params (role.id=8e9576d16f194938b86a1d1e087895bd, scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3, user.id=a45fb43467b9498eb1b647735d0b88b5) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:51.647393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7f5ec-173c-4c83-9513-779866c093b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:51.648822 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7f5ec-173c-4c83-9513-779866c093b6 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:51.650283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570f13d-4cd3-41ba-9f7a-48b1a0e45a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:51.650921 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570f13d-4cd3-41ba-9f7a-48b1a0e45a96 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:51.651200 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570f13d-4cd3-41ba-9f7a-48b1a0e45a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:51.651477 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570f13d-4cd3-41ba-9f7a-48b1a0e45a96 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:51.657936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 100/192] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:08:51 2022] GET /identity/v3/role_assignments?user.id=a45fb43467b9498eb1b647735d0b88b5&scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3&role.id=8e9576d16f194938b86a1d1e087895bd => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:51.657936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7f5ec-173c-4c83-9513-779866c093b6 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:51.666198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570f13d-4cd3-41ba-9f7a-48b1a0e45a96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:51.669337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:51.671695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-2570f13d-4cd3-41ba-9f7a-48b1a0e45a96 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:08:51.671695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:51.671695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:51.671695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:51.671695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:51.671695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:51.671695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:51.671695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:51.671695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:51.671695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:51.671695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:51.671695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:51.671695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:51.671695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:51.671695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:51.671695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:51.673009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:51.673009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:51.673009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:51.673009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:51.673009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:51.673009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:51.673009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:51.673009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:51.673009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:51.673009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:51.673009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:51.673009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:51.673009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:51.673009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:51.673009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:51.673009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:51.673869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:51.673869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:51.673869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:51.673869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:51.673869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:51.673869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:51.673869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:51.673869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:51.673869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:51.673869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:51.673869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:08:51.673869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:08:51.678837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 93/193] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:08:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:51.679335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570f13d-4cd3-41ba-9f7a-48b1a0e45a96 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:51.779275 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7f5ec-173c-4c83-9513-779866c093b6 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:51.804796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570f13d-4cd3-41ba-9f7a-48b1a0e45a96 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:51.819334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafb194-5f9a-4f08-8527-7fdf6b687f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:51.820102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafb194-5f9a-4f08-8527-7fdf6b687f29 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:51.820443 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafb194-5f9a-4f08-8527-7fdf6b687f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:51.820983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafb194-5f9a-4f08-8527-7fdf6b687f29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:51.850404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f27c0-cffe-4ec3-a2ad-bb81af1760af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:51.851491 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f27c0-cffe-4ec3-a2ad-bb81af1760af None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:51.852428 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f27c0-cffe-4ec3-a2ad-bb81af1760af None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:51.852428 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f27c0-cffe-4ec3-a2ad-bb81af1760af None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:51.853558 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f27c0-cffe-4ec3-a2ad-bb81af1760af None admin] RBAC: Adding query filter params (name=reader) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:51.854073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f27c0-cffe-4ec3-a2ad-bb81af1760af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:51.856466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f27c0-cffe-4ec3-a2ad-bb81af1760af None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:51.858766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:51.872811 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 94/194] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:08:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:51.874174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f27c0-cffe-4ec3-a2ad-bb81af1760af None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:51.902999 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafb194-5f9a-4f08-8527-7fdf6b687f29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:51.904459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aafb194-5f9a-4f08-8527-7fdf6b687f29 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:51.917949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:51.979648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f27c0-cffe-4ec3-a2ad-bb81af1760af None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:52.006553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 101/195] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:08:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:08:52.007605 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafb194-5f9a-4f08-8527-7fdf6b687f29 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:52.014185 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac567c-af1b-46c7-b4a6-bcee18466088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:52.014810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac567c-af1b-46c7-b4a6-bcee18466088 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:52.015084 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac567c-af1b-46c7-b4a6-bcee18466088 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:52.015692 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac567c-af1b-46c7-b4a6-bcee18466088 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:52.016491 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac567c-af1b-46c7-b4a6-bcee18466088 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:52.017943 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac567c-af1b-46c7-b4a6-bcee18466088 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:52.019650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 95/196] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:08:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:52.020395 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac567c-af1b-46c7-b4a6-bcee18466088 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:52.081792 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:52.109952 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aafb194-5f9a-4f08-8527-7fdf6b687f29 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:52.119704 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac567c-af1b-46c7-b4a6-bcee18466088 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:52.121392 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:52.145222 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c620e-d89d-4cec-8cd9-f42be9134d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:52.145855 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c620e-d89d-4cec-8cd9-f42be9134d6a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:52.146093 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c620e-d89d-4cec-8cd9-f42be9134d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:52.146358 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c620e-d89d-4cec-8cd9-f42be9134d6a None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:52.147012 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c620e-d89d-4cec-8cd9-f42be9134d6a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:52.147308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c620e-d89d-4cec-8cd9-f42be9134d6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:52.148670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c620e-d89d-4cec-8cd9-f42be9134d6a None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:52.153514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f046b-6849-46d9-bc19-74cb1c0ad9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:52.154168 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f046b-6849-46d9-bc19-74cb1c0ad9f6 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:52.154448 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f046b-6849-46d9-bc19-74cb1c0ad9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:52.154718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f046b-6849-46d9-bc19-74cb1c0ad9f6 None admin] PATH_INFO: `/v3/users/17f1c3d7516145758cee5edf5ebc5888` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:52.155838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f046b-6849-46d9-bc19-74cb1c0ad9f6 None admin] RBAC: Authorizing `identity:get_user(user_id=17f1c3d7516145758cee5edf5ebc5888)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:52.160633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 102/197] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:08:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:52.160633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c620e-d89d-4cec-8cd9-f42be9134d6a None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:52.160870 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f046b-6849-46d9-bc19-74cb1c0ad9f6 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:52.160870 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 96/198] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:52 2022] GET /identity/v3/users/17f1c3d7516145758cee5edf5ebc5888 => generated 289 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:52.160870 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f046b-6849-46d9-bc19-74cb1c0ad9f6 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:52.261296 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c620e-d89d-4cec-8cd9-f42be9134d6a None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:52.264464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f046b-6849-46d9-bc19-74cb1c0ad9f6 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:52.294814 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5cb7d1-94fd-4220-b181-b368a8568edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:52.295148 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:52.295446 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cb7d1-94fd-4220-b181-b368a8568edf None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:52.296097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cb7d1-94fd-4220-b181-b368a8568edf None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:52.296383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cb7d1-94fd-4220-b181-b368a8568edf None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:52.297776 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5cb7d1-94fd-4220-b181-b368a8568edf None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:52.299269 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5cb7d1-94fd-4220-b181-b368a8568edf None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:52.300542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a48e3b-1104-493b-8698-2ded704b64b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:52.301144 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a48e3b-1104-493b-8698-2ded704b64b8 None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:52.301271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 103/199] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:52 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:52.301442 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a48e3b-1104-493b-8698-2ded704b64b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:52.301705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a48e3b-1104-493b-8698-2ded704b64b8 None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:52.301950 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5cb7d1-94fd-4220-b181-b368a8568edf None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:52.302340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a48e3b-1104-493b-8698-2ded704b64b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:52.303745 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a48e3b-1104-493b-8698-2ded704b64b8 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:52.323963 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 97/200] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:08:52 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:08:52.324813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a48e3b-1104-493b-8698-2ded704b64b8 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:52.363171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:52.433962 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5cb7d1-94fd-4220-b181-b368a8568edf None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:52.450452 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a48e3b-1104-493b-8698-2ded704b64b8 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:52.472247 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b7733-6ce5-48dd-9fb5-5e7601cd7602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:52.472943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b7733-6ce5-48dd-9fb5-5e7601cd7602 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:52.473263 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b7733-6ce5-48dd-9fb5-5e7601cd7602 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:52.473535 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b7733-6ce5-48dd-9fb5-5e7601cd7602 None admin] PATH_INFO: `/v3/users/d00c15ca137346aea94f5698900fe15c` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:52.474712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b7733-6ce5-48dd-9fb5-5e7601cd7602 None admin] RBAC: Authorizing `identity:get_user(user_id=d00c15ca137346aea94f5698900fe15c)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:52.476169 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b7733-6ce5-48dd-9fb5-5e7601cd7602 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:52.482661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 104/201] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:52 2022] GET /identity/v3/users/d00c15ca137346aea94f5698900fe15c => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:52.482661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b7733-6ce5-48dd-9fb5-5e7601cd7602 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:52.497281 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549ebda-48fb-45cc-bd83-2f2b13af8172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:52.497926 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549ebda-48fb-45cc-bd83-2f2b13af8172 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:52.498441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549ebda-48fb-45cc-bd83-2f2b13af8172 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:52.498709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549ebda-48fb-45cc-bd83-2f2b13af8172 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:52.561407 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:52.578524 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549ebda-48fb-45cc-bd83-2f2b13af8172 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:52.579749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549ebda-48fb-45cc-bd83-2f2b13af8172 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:52.604419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b7733-6ce5-48dd-9fb5-5e7601cd7602 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:52.614968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:52.651178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2c8b0-3ee0-4cd5-be46-866585209c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:52.651800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2c8b0-3ee0-4cd5-be46-866585209c12 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:52.652088 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2c8b0-3ee0-4cd5-be46-866585209c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:52.652479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2c8b0-3ee0-4cd5-be46-866585209c12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:52.653256 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf2c8b0-3ee0-4cd5-be46-866585209c12 None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=81a7733885564252af45b01ea11bfed2, user.id=d00c15ca137346aea94f5698900fe15c) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:52.653574 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf2c8b0-3ee0-4cd5-be46-866585209c12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:52.655132 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf2c8b0-3ee0-4cd5-be46-866585209c12 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:52.663403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 105/202] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:08:52 2022] GET /identity/v3/role_assignments?user.id=d00c15ca137346aea94f5698900fe15c&scope.project.id=81a7733885564252af45b01ea11bfed2&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:52.664235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2c8b0-3ee0-4cd5-be46-866585209c12 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:52.707546 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 98/203] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:08:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:08:52.715013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549ebda-48fb-45cc-bd83-2f2b13af8172 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:52.785810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2c8b0-3ee0-4cd5-be46-866585209c12 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:52.815600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:52.829382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba570d-bd53-47e3-9085-d9f01aea1c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:52.829940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549ebda-48fb-45cc-bd83-2f2b13af8172 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:52.830072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba570d-bd53-47e3-9085-d9f01aea1c8f None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:52.830276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba570d-bd53-47e3-9085-d9f01aea1c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:52.831075 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba570d-bd53-47e3-9085-d9f01aea1c8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:52.868822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394d390-45b3-4631-9ab1-868e98e33cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:52.869529 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394d390-45b3-4631-9ab1-868e98e33cae None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:52.869777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394d390-45b3-4631-9ab1-868e98e33cae None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:52.870042 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394d390-45b3-4631-9ab1-868e98e33cae None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:52.874835 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394d390-45b3-4631-9ab1-868e98e33cae None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:52.876435 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394d390-45b3-4631-9ab1-868e98e33cae None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:52.878719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 99/204] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:52 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:52.884378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:52.887506 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394d390-45b3-4631-9ab1-868e98e33cae None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:52.923529 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba570d-bd53-47e3-9085-d9f01aea1c8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:52.925744 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba570d-bd53-47e3-9085-d9f01aea1c8f None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:53.047620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394d390-45b3-4631-9ab1-868e98e33cae None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:53.088691 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe8a2f-5e22-455e-9944-a3474f4d5ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:53.090201 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe8a2f-5e22-455e-9944-a3474f4d5ffe None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:53.090479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe8a2f-5e22-455e-9944-a3474f4d5ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:53.090749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe8a2f-5e22-455e-9944-a3474f4d5ffe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:53.091485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe8a2f-5e22-455e-9944-a3474f4d5ffe None admin] RBAC: Adding query filter params (role.id=edf27bedabd24681b24ec2a72a774a9c, scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3, user.id=17f1c3d7516145758cee5edf5ebc5888) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:53.091795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe8a2f-5e22-455e-9944-a3474f4d5ffe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:53.093255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe8a2f-5e22-455e-9944-a3474f4d5ffe None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:53.107016 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 100/205] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:08:52 2022] GET /identity/v3/role_assignments?user.id=17f1c3d7516145758cee5edf5ebc5888&scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3&role.id=edf27bedabd24681b24ec2a72a774a9c => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:53.140603 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 106/206] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:08:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:08:53.147276 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe8a2f-5e22-455e-9944-a3474f4d5ffe None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:53.195061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:53.321351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe8a2f-5e22-455e-9944-a3474f4d5ffe None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:53.362964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b77836-f4d2-4a20-84f5-17ce8ea5b866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:53.363605 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b77836-f4d2-4a20-84f5-17ce8ea5b866 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:53.363887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b77836-f4d2-4a20-84f5-17ce8ea5b866 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:53.364162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b77836-f4d2-4a20-84f5-17ce8ea5b866 None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:53.366630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b77836-f4d2-4a20-84f5-17ce8ea5b866 None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:53.368077 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b77836-f4d2-4a20-84f5-17ce8ea5b866 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:53.369731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 101/207] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:53 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:53.378136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba570d-bd53-47e3-9085-d9f01aea1c8f None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:53.454947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:53.487876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba570d-bd53-47e3-9085-d9f01aea1c8f None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:53.529602 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11108324-c796-4273-b12c-4c64430b6672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:53.530225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11108324-c796-4273-b12c-4c64430b6672 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:53.530509 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11108324-c796-4273-b12c-4c64430b6672 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:53.530849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11108324-c796-4273-b12c-4c64430b6672 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:53.531597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11108324-c796-4273-b12c-4c64430b6672 None admin] RBAC: Adding query filter params (role.id=8e9576d16f194938b86a1d1e087895bd, scope.project.id=81a7733885564252af45b01ea11bfed2, user.id=d00c15ca137346aea94f5698900fe15c) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:53.531908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11108324-c796-4273-b12c-4c64430b6672 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:53.533428 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11108324-c796-4273-b12c-4c64430b6672 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:53.541554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 107/208] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:08:53 2022] GET /identity/v3/role_assignments?user.id=d00c15ca137346aea94f5698900fe15c&scope.project.id=81a7733885564252af45b01ea11bfed2&role.id=8e9576d16f194938b86a1d1e087895bd => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:55.021669 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765a47b-0a91-4257-8366-125bbd3acded None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:55.022219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765a47b-0a91-4257-8366-125bbd3acded None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:55.022219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765a47b-0a91-4257-8366-125bbd3acded None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:55.023548 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 102/209] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:55.029001 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0896779-4d3c-4a22-8a6f-2dd886e4db18 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:55.029550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0896779-4d3c-4a22-8a6f-2dd886e4db18 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:55.029850 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0896779-4d3c-4a22-8a6f-2dd886e4db18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:55.055406 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e0896779-4d3c-4a22-8a6f-2dd886e4db18 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:55.055613 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a943316-36b6-4e67-a71e-21de5193f1ed None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:55.055613 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a943316-36b6-4e67-a71e-21de5193f1ed None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:55.055613 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a943316-36b6-4e67-a71e-21de5193f1ed None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:55.056675 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e0896779-4d3c-4a22-8a6f-2dd886e4db18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:55.056923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 103/210] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:55.062163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb7a86-ae86-4f0c-b035-3264797eccb4 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:55.062444 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb7a86-ae86-4f0c-b035-3264797eccb4 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:55.062722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb7a86-ae86-4f0c-b035-3264797eccb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:55.086633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-5ceb7a86-ae86-4f0c-b035-3264797eccb4 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:55.088034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5ceb7a86-ae86-4f0c-b035-3264797eccb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:55.134032 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 108/211] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:55.135481 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0024ec4-3785-4fd9-85e5-8f4940b2b225 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:55.135835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0024ec4-3785-4fd9-85e5-8f4940b2b225 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:55.136177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0024ec4-3785-4fd9-85e5-8f4940b2b225 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:55.137537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 109/212] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:55.142687 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bed704-53d7-49eb-8fdb-5d999bbbd799 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:55.143072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bed704-53d7-49eb-8fdb-5d999bbbd799 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:55.143863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bed704-53d7-49eb-8fdb-5d999bbbd799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:55.161145 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-d5bed704-53d7-49eb-8fdb-5d999bbbd799 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:55.162454 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d5bed704-53d7-49eb-8fdb-5d999bbbd799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:55.167615 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 104/213] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:55.232563 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:55.236467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 110/214] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:08:55.405039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb7a86-ae86-4f0c-b035-3264797eccb4 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:55.424555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:55.462847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bed704-53d7-49eb-8fdb-5d999bbbd799 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:55.524917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb7a86-ae86-4f0c-b035-3264797eccb4 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:55.567283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e3c28-b198-41d6-8299-396988142f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:55.568324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e3c28-b198-41d6-8299-396988142f46 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:55.568849 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e3c28-b198-41d6-8299-396988142f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:55.569156 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e3c28-b198-41d6-8299-396988142f46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:55.570659 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bed704-53d7-49eb-8fdb-5d999bbbd799 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:55.581293 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e3c28-b198-41d6-8299-396988142f46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:55.586066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-c17e3c28-b198-41d6-8299-396988142f46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:08:55.586066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:55.586066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:55.586066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:55.586066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:55.586066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:55.586066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:55.586066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:55.586066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:55.586066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:55.586066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:55.586066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:55.586066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:55.586066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:55.586066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:55.586066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:55.586924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:55.586924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:55.586924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:55.586924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:55.586924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:55.586924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:55.586924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:55.586924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:55.586924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:55.586924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:55.586924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:55.586924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:55.586924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:55.586924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:55.586924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:55.586924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:55.587707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:55.587707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:55.587707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:55.587707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:55.587707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:55.587707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:55.587707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:55.587707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:55.587707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:55.587707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:55.587707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:08:55.587707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:08:55.587707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 105/215] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:08:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:55.588984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e3c28-b198-41d6-8299-396988142f46 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:55.608279 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45486f3-4b28-4320-9fbb-dc51e91a7263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:55.608913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45486f3-4b28-4320-9fbb-dc51e91a7263 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:55.609219 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45486f3-4b28-4320-9fbb-dc51e91a7263 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:55.609495 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45486f3-4b28-4320-9fbb-dc51e91a7263 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:55.620052 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45486f3-4b28-4320-9fbb-dc51e91a7263 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:55.624897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-f45486f3-4b28-4320-9fbb-dc51e91a7263 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:08:55.624897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:55.624897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:55.624897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:55.624897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:55.624897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:55.624897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:55.624897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:55.624897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:55.624897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:55.624897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:55.624897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:55.624897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:55.624897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:55.624897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:55.624897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:55.625939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:55.625939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:55.625939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:55.625939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:55.625939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:55.625939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:55.625939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:55.625939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:55.625939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:55.625939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:55.625939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:55.625939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:55.625939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:55.625939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:55.625939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:55.625939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:55.631874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:55.631874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:55.631874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:55.631874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:55.631874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:55.631874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:55.631874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:55.631874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:55.631874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:55.631874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:55.631874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:08:55.631874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:08:55.631874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 111/216] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:08:55.631874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45486f3-4b28-4320-9fbb-dc51e91a7263 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:55.685874 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:55.691599 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:55.710803 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e3c28-b198-41d6-8299-396988142f46 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:55.747474 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45486f3-4b28-4320-9fbb-dc51e91a7263 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:55.755293 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1c7ad-2b1e-4840-a495-a36d76dee5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:55.756719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1c7ad-2b1e-4840-a495-a36d76dee5ac None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:55.757246 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1c7ad-2b1e-4840-a495-a36d76dee5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:55.758148 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1c7ad-2b1e-4840-a495-a36d76dee5ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:55.759856 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1c7ad-2b1e-4840-a495-a36d76dee5ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:55.762112 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1c7ad-2b1e-4840-a495-a36d76dee5ac None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:55.764641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 106/217] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:08:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:55.765978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1c7ad-2b1e-4840-a495-a36d76dee5ac None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:55.784505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afb7e8-93da-4d51-aef6-ec751048dadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:55.785357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afb7e8-93da-4d51-aef6-ec751048dadc None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:55.785636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afb7e8-93da-4d51-aef6-ec751048dadc None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:55.785910 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afb7e8-93da-4d51-aef6-ec751048dadc None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:55.786709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4afb7e8-93da-4d51-aef6-ec751048dadc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:55.786960 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4afb7e8-93da-4d51-aef6-ec751048dadc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:55.788419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4afb7e8-93da-4d51-aef6-ec751048dadc None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:55.797764 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 112/218] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:55.802007 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afb7e8-93da-4d51-aef6-ec751048dadc None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:55.877638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1c7ad-2b1e-4840-a495-a36d76dee5ac None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:55.903076 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afb7e8-93da-4d51-aef6-ec751048dadc None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:55.911673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627342d-29f1-4792-9780-c61a3e734202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:55.912277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627342d-29f1-4792-9780-c61a3e734202 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:55.912554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627342d-29f1-4792-9780-c61a3e734202 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:55.912825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627342d-29f1-4792-9780-c61a3e734202 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:55.913517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627342d-29f1-4792-9780-c61a3e734202 None admin] RBAC: Adding query filter params (name=member) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:55.913817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627342d-29f1-4792-9780-c61a3e734202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:55.915199 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627342d-29f1-4792-9780-c61a3e734202 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:55.920967 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:55.923415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 107/219] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:08:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:55.925972 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627342d-29f1-4792-9780-c61a3e734202 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:55.937157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:55.939431 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175fd53-82a7-462d-a165-c43bad3441d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:55.940074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175fd53-82a7-462d-a165-c43bad3441d8 None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:55.940361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175fd53-82a7-462d-a165-c43bad3441d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:55.940637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175fd53-82a7-462d-a165-c43bad3441d8 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:55.941318 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175fd53-82a7-462d-a165-c43bad3441d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:55.942792 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175fd53-82a7-462d-a165-c43bad3441d8 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:55.963861 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 113/220] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:08:55 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:08:55.966601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57df981-7988-425c-8a01-80e66b66593a None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:55.967049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57df981-7988-425c-8a01-80e66b66593a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:55.967333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57df981-7988-425c-8a01-80e66b66593a None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:55.968748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 114/221] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:55.969683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57df981-7988-425c-8a01-80e66b66593a None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:56.039649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627342d-29f1-4792-9780-c61a3e734202 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:56.076944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762ec58-a799-4b7a-a6b6-13a35ae90f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:56.077640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762ec58-a799-4b7a-a6b6-13a35ae90f9b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:56.077955 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762ec58-a799-4b7a-a6b6-13a35ae90f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:56.078279 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762ec58-a799-4b7a-a6b6-13a35ae90f9b None admin] PATH_INFO: `/v3/users/d00c15ca137346aea94f5698900fe15c` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:56.079565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762ec58-a799-4b7a-a6b6-13a35ae90f9b None admin] RBAC: Authorizing `identity:get_user(user_id=d00c15ca137346aea94f5698900fe15c)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:56.082200 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762ec58-a799-4b7a-a6b6-13a35ae90f9b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:56.084513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 108/222] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:55 2022] GET /identity/v3/users/d00c15ca137346aea94f5698900fe15c => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:56.086196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd7590-9829-4e30-9da7-fd820a9055a8 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:56.086485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd7590-9829-4e30-9da7-fd820a9055a8 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:56.086766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd7590-9829-4e30-9da7-fd820a9055a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:56.088391 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57df981-7988-425c-8a01-80e66b66593a None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:56.107268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-eddd7590-9829-4e30-9da7-fd820a9055a8 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:56.108618 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-eddd7590-9829-4e30-9da7-fd820a9055a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:56.125947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846aeca-1a22-4125-a2d6-920f3b6aaefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:56.126631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846aeca-1a22-4125-a2d6-920f3b6aaefb None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:56.126871 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846aeca-1a22-4125-a2d6-920f3b6aaefb None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:56.127145 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846aeca-1a22-4125-a2d6-920f3b6aaefb None admin] PATH_INFO: `/v3/users/0555244f65384fd5a9d967f604e5d328` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:56.143794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846aeca-1a22-4125-a2d6-920f3b6aaefb None admin] RBAC: Authorizing `identity:get_user(user_id=0555244f65384fd5a9d967f604e5d328)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:56.145290 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846aeca-1a22-4125-a2d6-920f3b6aaefb None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:56.146976 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 115/223] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:55 2022] GET /identity/v3/users/0555244f65384fd5a9d967f604e5d328 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:56.147736 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846aeca-1a22-4125-a2d6-920f3b6aaefb None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:56.190541 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:56.192790 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 109/224] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:56.193841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd7590-9829-4e30-9da7-fd820a9055a8 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:56.202790 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:56.293751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846aeca-1a22-4125-a2d6-920f3b6aaefb None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:56.321851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd7590-9829-4e30-9da7-fd820a9055a8 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:56.342534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cba515-1725-4c04-b149-4a602100fa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:56.343344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cba515-1725-4c04-b149-4a602100fa29 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:56.343626 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cba515-1725-4c04-b149-4a602100fa29 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:56.344226 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cba515-1725-4c04-b149-4a602100fa29 None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:56.345431 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cba515-1725-4c04-b149-4a602100fa29 None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:56.347371 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cba515-1725-4c04-b149-4a602100fa29 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:56.349314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 116/225] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:56 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:56.354759 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cba515-1725-4c04-b149-4a602100fa29 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:56.363604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb13b7-894a-47fb-a2bb-e73f82afa7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:56.364221 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb13b7-894a-47fb-a2bb-e73f82afa7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:56.364499 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb13b7-894a-47fb-a2bb-e73f82afa7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:56.364776 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb13b7-894a-47fb-a2bb-e73f82afa7b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:56.446573 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb13b7-894a-47fb-a2bb-e73f82afa7b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:56.447965 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb13b7-894a-47fb-a2bb-e73f82afa7b7 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:56.457146 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:56.485510 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:56.489258 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cba515-1725-4c04-b149-4a602100fa29 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:56.536802 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48b9df-f05c-4e36-88ed-7810fffc16b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:56.537554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48b9df-f05c-4e36-88ed-7810fffc16b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:56.537837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48b9df-f05c-4e36-88ed-7810fffc16b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:56.538112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48b9df-f05c-4e36-88ed-7810fffc16b6 None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2/users/d00c15ca137346aea94f5698900fe15c/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:56.540133 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48b9df-f05c-4e36-88ed-7810fffc16b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=81a7733885564252af45b01ea11bfed2, user_id=d00c15ca137346aea94f5698900fe15c, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:56.541644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48b9df-f05c-4e36-88ed-7810fffc16b6 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:56.560725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 117/226] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:08:56 2022] PUT /identity/v3/projects/81a7733885564252af45b01ea11bfed2/users/d00c15ca137346aea94f5698900fe15c/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:08:56.561594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48b9df-f05c-4e36-88ed-7810fffc16b6 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:56.590271 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 110/227] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:08:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:08:56.599845 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb13b7-894a-47fb-a2bb-e73f82afa7b7 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:56.772289 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48b9df-f05c-4e36-88ed-7810fffc16b6 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:56.806347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb13b7-894a-47fb-a2bb-e73f82afa7b7 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:56.816333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:56.821520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab438ff0-e632-4fcb-84ed-a022ee551c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:56.822312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab438ff0-e632-4fcb-84ed-a022ee551c9d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:56.822643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab438ff0-e632-4fcb-84ed-a022ee551c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:56.822920 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab438ff0-e632-4fcb-84ed-a022ee551c9d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:56.838242 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab438ff0-e632-4fcb-84ed-a022ee551c9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:56.844039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-ab438ff0-e632-4fcb-84ed-a022ee551c9d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:08:56.844039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:56.844039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:56.844039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:56.844039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:56.844039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:56.844039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:56.844039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:56.844039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:56.844039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:56.844039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:56.844039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:56.844039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:56.844039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:56.844039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:56.844039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:56.844926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:56.844926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:56.844926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:56.844926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:56.844926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:56.844926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:56.844926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:56.844926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:56.844926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:56.844926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:56.844926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:56.844926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:56.844926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:56.844926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:56.844926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:56.844926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:56.845750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:56.845750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:56.845750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:56.845750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:56.845750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:56.845750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:56.845750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:56.845750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:56.845750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:56.845750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:56.845750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:08:56.845750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:08:56.845750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 118/228] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:08:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:56.852978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab438ff0-e632-4fcb-84ed-a022ee551c9d None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:56.858605 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda27b83-1ca4-48ee-9ca2-6f86a7baa393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:56.859469 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda27b83-1ca4-48ee-9ca2-6f86a7baa393 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:56.859941 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda27b83-1ca4-48ee-9ca2-6f86a7baa393 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:56.860427 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda27b83-1ca4-48ee-9ca2-6f86a7baa393 None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:56.862378 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda27b83-1ca4-48ee-9ca2-6f86a7baa393 None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:56.864043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda27b83-1ca4-48ee-9ca2-6f86a7baa393 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:56.866195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 111/229] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:56 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:56.872086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda27b83-1ca4-48ee-9ca2-6f86a7baa393 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:56.897524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:57.028670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab438ff0-e632-4fcb-84ed-a022ee551c9d None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:57.071223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda27b83-1ca4-48ee-9ca2-6f86a7baa393 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:57.099715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd403b10-3197-44a6-a057-97802f5e8983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:57.101016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd403b10-3197-44a6-a057-97802f5e8983 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:57.102282 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd403b10-3197-44a6-a057-97802f5e8983 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:57.102745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd403b10-3197-44a6-a057-97802f5e8983 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:57.105051 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd403b10-3197-44a6-a057-97802f5e8983 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:57.105717 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd403b10-3197-44a6-a057-97802f5e8983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:57.107587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd403b10-3197-44a6-a057-97802f5e8983 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:57.118853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 119/230] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:08:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:57.123874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd403b10-3197-44a6-a057-97802f5e8983 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:57.141565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762c568-fd8e-4b62-9b45-48d51ee1bdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:57.142401 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762c568-fd8e-4b62-9b45-48d51ee1bdd0 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:57.142684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762c568-fd8e-4b62-9b45-48d51ee1bdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:57.143507 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a762c568-fd8e-4b62-9b45-48d51ee1bdd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:57.144233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a762c568-fd8e-4b62-9b45-48d51ee1bdd0 None admin] RBAC: Adding query filter params (role.id=2201e005d5ef449896a0e68a940678ec, scope.project.id=81a7733885564252af45b01ea11bfed2, user.id=0555244f65384fd5a9d967f604e5d328) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:57.144546 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a762c568-fd8e-4b62-9b45-48d51ee1bdd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:57.146012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a762c568-fd8e-4b62-9b45-48d51ee1bdd0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:57.157864 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 112/231] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:08:56 2022] GET /identity/v3/role_assignments?user.id=0555244f65384fd5a9d967f604e5d328&scope.project.id=81a7733885564252af45b01ea11bfed2&role.id=2201e005d5ef449896a0e68a940678ec => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:57.244328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:57.264700 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd403b10-3197-44a6-a057-97802f5e8983 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:57.303943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee161a5-36a2-43c3-b056-8d09b7ab5691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:57.304519 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee161a5-36a2-43c3-b056-8d09b7ab5691 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:57.304806 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee161a5-36a2-43c3-b056-8d09b7ab5691 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:57.305084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee161a5-36a2-43c3-b056-8d09b7ab5691 None admin] PATH_INFO: `/v3/users/d00c15ca137346aea94f5698900fe15c` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:57.306429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee161a5-36a2-43c3-b056-8d09b7ab5691 None admin] RBAC: Authorizing `identity:get_user(user_id=d00c15ca137346aea94f5698900fe15c)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:57.307902 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee161a5-36a2-43c3-b056-8d09b7ab5691 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:57.309803 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 120/232] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:57 2022] GET /identity/v3/users/d00c15ca137346aea94f5698900fe15c => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:57.315478 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762c568-fd8e-4b62-9b45-48d51ee1bdd0 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:57.366222 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:57.438474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a762c568-fd8e-4b62-9b45-48d51ee1bdd0 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:57.490934 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8cc3b-a37a-4585-aba6-9748a991763b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:57.491839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8cc3b-a37a-4585-aba6-9748a991763b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:57.492288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8cc3b-a37a-4585-aba6-9748a991763b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:57.492719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8cc3b-a37a-4585-aba6-9748a991763b None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:57.494066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8cc3b-a37a-4585-aba6-9748a991763b None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:57.495682 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8cc3b-a37a-4585-aba6-9748a991763b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:57.497526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 113/233] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:08:57 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:57.506284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee161a5-36a2-43c3-b056-8d09b7ab5691 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:57.624627 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee161a5-36a2-43c3-b056-8d09b7ab5691 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:57.662407 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724482a-3c4f-4b95-bad8-392f21f4b9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:57.663086 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724482a-3c4f-4b95-bad8-392f21f4b9c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:57.663370 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724482a-3c4f-4b95-bad8-392f21f4b9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:57.663648 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724482a-3c4f-4b95-bad8-392f21f4b9c4 None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2/users/d00c15ca137346aea94f5698900fe15c/roles/8e9576d16f194938b86a1d1e087895bd` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:57.665906 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724482a-3c4f-4b95-bad8-392f21f4b9c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=81a7733885564252af45b01ea11bfed2, user_id=d00c15ca137346aea94f5698900fe15c, role_id=8e9576d16f194938b86a1d1e087895bd)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:57.667369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724482a-3c4f-4b95-bad8-392f21f4b9c4 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:57.682402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 121/234] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:08:57 2022] PUT /identity/v3/projects/81a7733885564252af45b01ea11bfed2/users/d00c15ca137346aea94f5698900fe15c/roles/8e9576d16f194938b86a1d1e087895bd => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:08:58.617045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dc6a1-5d5b-4054-a97e-714d0c9d54f0 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:58.617611 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dc6a1-5d5b-4054-a97e-714d0c9d54f0 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:58.617611 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dc6a1-5d5b-4054-a97e-714d0c9d54f0 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:58.619226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 114/235] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:58.628479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fabaf-30c9-4e69-9554-8a619f10515f None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:58.628896 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fabaf-30c9-4e69-9554-8a619f10515f None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:58.629079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fabaf-30c9-4e69-9554-8a619f10515f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:58.648158 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-0f1fabaf-30c9-4e69-9554-8a619f10515f None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:58.649568 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0f1fabaf-30c9-4e69-9554-8a619f10515f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:58.652542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:58.733813 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 122/236] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:58.869552 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3c0e8-7bae-4bf5-9044-31e9bcbf399e None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:58.870160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3c0e8-7bae-4bf5-9044-31e9bcbf399e None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:58.870930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb3c0e8-7bae-4bf5-9044-31e9bcbf399e None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:58.872733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 115/237] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:58.878411 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b6cb6-c6af-4d66-bd98-e3e3c6b1dd81 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:58.878968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b6cb6-c6af-4d66-bd98-e3e3c6b1dd81 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:58.879471 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b6cb6-c6af-4d66-bd98-e3e3c6b1dd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:58.898892 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-b19b6cb6-c6af-4d66-bd98-e3e3c6b1dd81 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:58.900075 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b19b6cb6-c6af-4d66-bd98-e3e3c6b1dd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:58.973366 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 123/238] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:59.079515 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3c0e8-7bae-4bf5-9044-31e9bcbf399e None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:59.214538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb3c0e8-7bae-4bf5-9044-31e9bcbf399e None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:59.226472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:59.265797 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff697e-f1a7-4baa-b4df-2ed7ed7e61d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:59.266436 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff697e-f1a7-4baa-b4df-2ed7ed7e61d7 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:59.266721 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff697e-f1a7-4baa-b4df-2ed7ed7e61d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:59.266995 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff697e-f1a7-4baa-b4df-2ed7ed7e61d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:59.277268 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b6cb6-c6af-4d66-bd98-e3e3c6b1dd81 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:59.287395 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff697e-f1a7-4baa-b4df-2ed7ed7e61d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:08:59.293714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-06ff697e-f1a7-4baa-b4df-2ed7ed7e61d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:08:59.293714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:08:59.293714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:08:59.293714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:08:59.293714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:08:59.293714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:08:59.293714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:08:59.293714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:08:59.293714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:08:59.293714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:08:59.293714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:08:59.293714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:08:59.293714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:08:59.293714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:08:59.293714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:08:59.293714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:08:59.294587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:08:59.294587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:08:59.294587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:08:59.294587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:08:59.294587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:08:59.294587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:08:59.294587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:08:59.294587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:08:59.294587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:08:59.294587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:08:59.294587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:08:59.294587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:08:59.294587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:08:59.294587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:08:59.294587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:08:59.294587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:08:59.300334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:08:59.300334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:08:59.300334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:08:59.300334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:08:59.300334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:08:59.300334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:08:59.300334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:08:59.300334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:08:59.300334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:08:59.300334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:08:59.300334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:08:59.300334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:08:59.300334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 116/239] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:08:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:08:59.303127 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff697e-f1a7-4baa-b4df-2ed7ed7e61d7 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:59.398352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:59.420163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b6cb6-c6af-4d66-bd98-e3e3c6b1dd81 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:59.431206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff697e-f1a7-4baa-b4df-2ed7ed7e61d7 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:59.467936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67ed621-4fb5-46cc-b7c1-acdddc29ed9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:59.468845 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ed621-4fb5-46cc-b7c1-acdddc29ed9c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:59.469289 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ed621-4fb5-46cc-b7c1-acdddc29ed9c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:59.470016 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac889f-680e-46b8-9dc5-a459577886e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:59.470485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ed621-4fb5-46cc-b7c1-acdddc29ed9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:59.472395 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac889f-680e-46b8-9dc5-a459577886e5 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:59.472849 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac889f-680e-46b8-9dc5-a459577886e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:59.473279 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac889f-680e-46b8-9dc5-a459577886e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:59.473890 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ed621-4fb5-46cc-b7c1-acdddc29ed9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:59.474231 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac889f-680e-46b8-9dc5-a459577886e5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:08:59.474665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac889f-680e-46b8-9dc5-a459577886e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:59.475648 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67ed621-4fb5-46cc-b7c1-acdddc29ed9c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:59.476324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac889f-680e-46b8-9dc5-a459577886e5 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:59.478055 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 124/240] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:08:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:08:59.479893 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e14f44-43b1-43bb-bfce-35ac6f5918b6 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:59.480280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e14f44-43b1-43bb-bfce-35ac6f5918b6 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:59.480654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e14f44-43b1-43bb-bfce-35ac6f5918b6 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:59.482096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 125/241] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:08:59.483464 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e14f44-43b1-43bb-bfce-35ac6f5918b6 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:59.486918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 117/242] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:08:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:59.494185 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac889f-680e-46b8-9dc5-a459577886e5 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:59.529463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:59.618041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e14f44-43b1-43bb-bfce-35ac6f5918b6 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:59.620299 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac889f-680e-46b8-9dc5-a459577886e5 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:59.655953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7beb3-853d-4aac-a868-d97c03470c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:59.657336 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7beb3-853d-4aac-a868-d97c03470c88 None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:59.657788 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7beb3-853d-4aac-a868-d97c03470c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:59.658236 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7beb3-853d-4aac-a868-d97c03470c88 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:59.659108 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7beb3-853d-4aac-a868-d97c03470c88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:59.660589 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93df01-ae55-4a98-9910-40c5b63dec72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:59.660956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a7beb3-853d-4aac-a868-d97c03470c88 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:59.661391 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93df01-ae55-4a98-9910-40c5b63dec72 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:59.661745 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93df01-ae55-4a98-9910-40c5b63dec72 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:59.662089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93df01-ae55-4a98-9910-40c5b63dec72 None admin] PATH_INFO: `/v3/users/c0c2f447c91b455db5c8e72b7497f907` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:59.672063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:08:59.680448 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93df01-ae55-4a98-9910-40c5b63dec72 None admin] RBAC: Authorizing `identity:get_user(user_id=c0c2f447c91b455db5c8e72b7497f907)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:08:59.683008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf93df01-ae55-4a98-9910-40c5b63dec72 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:08:59.685711 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 118/243] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:08:59 2022] GET /identity/v3/users/c0c2f447c91b455db5c8e72b7497f907 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:08:59.687097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 126/244] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:08:59 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:08:59.687347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a1fe3-9e65-4ebe-a4c9-7981e43db94a None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:59.687636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a1fe3-9e65-4ebe-a4c9-7981e43db94a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:59.687913 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a1fe3-9e65-4ebe-a4c9-7981e43db94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:08:59.702356 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7beb3-853d-4aac-a868-d97c03470c88 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:08:59.718587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-5f7a1fe3-9e65-4ebe-a4c9-7981e43db94a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:08:59.719984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5f7a1fe3-9e65-4ebe-a4c9-7981e43db94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:08:59.881990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 119/245] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:08:59.904192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7beb3-853d-4aac-a868-d97c03470c88 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:08:59.942302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02c592-558d-4666-9230-0663b20622ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:08:59.942924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02c592-558d-4666-9230-0663b20622ad None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:08:59.943211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02c592-558d-4666-9230-0663b20622ad None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:08:59.943489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02c592-558d-4666-9230-0663b20622ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:00.011771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:00.020433 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02c592-558d-4666-9230-0663b20622ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:00.022432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02c592-558d-4666-9230-0663b20622ad None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:00.139518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 127/246] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:08:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:00.144611 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a1fe3-9e65-4ebe-a4c9-7981e43db94a None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:00.175065 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:00.206500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02c592-558d-4666-9230-0663b20622ad None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:00.305793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a1fe3-9e65-4ebe-a4c9-7981e43db94a None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:00.334503 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:00.345610 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02c592-558d-4666-9230-0663b20622ad None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:00.353728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e676f-06bd-42a5-ad2a-f884ad36129d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:00.354383 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e676f-06bd-42a5-ad2a-f884ad36129d None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:00.354663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e676f-06bd-42a5-ad2a-f884ad36129d None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:00.354938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e676f-06bd-42a5-ad2a-f884ad36129d None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:00.356184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e676f-06bd-42a5-ad2a-f884ad36129d None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:00.357876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e676f-06bd-42a5-ad2a-f884ad36129d None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:00.359528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 120/247] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:00 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:00.361243 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d2973-c0cd-44d5-a6de-a06de37fe49e None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:00.362063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d2973-c0cd-44d5-a6de-a06de37fe49e None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:00.362323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d2973-c0cd-44d5-a6de-a06de37fe49e None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:00.364700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 121/248] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:00.369896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0d5b3-0703-48ee-b934-2d33b1ba7e24 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:00.370180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0d5b3-0703-48ee-b934-2d33b1ba7e24 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:00.370454 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0d5b3-0703-48ee-b934-2d33b1ba7e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:00.388431 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e3a74-e163-4e85-a813-cc810e514008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:00.389081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e3a74-e163-4e85-a813-cc810e514008 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:00.389399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e3a74-e163-4e85-a813-cc810e514008 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:00.389674 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e3a74-e163-4e85-a813-cc810e514008 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:00.391237 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-81f0d5b3-0703-48ee-b934-2d33b1ba7e24 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:00.392435 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-81f0d5b3-0703-48ee-b934-2d33b1ba7e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:00.401577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e3a74-e163-4e85-a813-cc810e514008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:00.407099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-f17e3a74-e163-4e85-a813-cc810e514008 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:09:00.407099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:00.407099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:00.407099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:00.407099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:00.407099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:00.407099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:00.407099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:00.407099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:00.407099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:00.407099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:00.407099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:00.407099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:00.407099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:00.407099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:00.407099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:00.407973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:00.407973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:00.407973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:00.407973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:00.407973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:00.407973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:00.407973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:00.407973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:00.407973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:00.407973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:00.407973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:00.407973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:00.407973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:00.407973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:00.407973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:00.407973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:00.409053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:00.409053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:00.409053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:00.409053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:00.409053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:00.409053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:00.409053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:00.409053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:00.409053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:00.409053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:00.409053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:09:00.409053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:00.411567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 128/249] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:09:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:00.412711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e3a74-e163-4e85-a813-cc810e514008 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:00.481900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 122/250] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:09:00.482489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0d5b3-0703-48ee-b934-2d33b1ba7e24 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:00.498527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:00.542306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e3a74-e163-4e85-a813-cc810e514008 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:00.590185 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51de26-fa83-4297-b181-481226525a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:00.591158 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51de26-fa83-4297-b181-481226525a44 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:00.591639 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51de26-fa83-4297-b181-481226525a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:00.591742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0d5b3-0703-48ee-b934-2d33b1ba7e24 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:00.592194 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51de26-fa83-4297-b181-481226525a44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:00.593389 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51de26-fa83-4297-b181-481226525a44 None admin] RBAC: Adding query filter params (role.id=edf27bedabd24681b24ec2a72a774a9c, scope.project.id=81a7733885564252af45b01ea11bfed2, user.id=c0c2f447c91b455db5c8e72b7497f907) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:00.594797 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51de26-fa83-4297-b181-481226525a44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:00.597341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51de26-fa83-4297-b181-481226525a44 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:00.604929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 129/251] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:09:00 2022] GET /identity/v3/role_assignments?user.id=c0c2f447c91b455db5c8e72b7497f907&scope.project.id=81a7733885564252af45b01ea11bfed2&role.id=edf27bedabd24681b24ec2a72a774a9c => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:00.606655 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e462c28-dd50-43f2-883f-4c42b3a32821 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:00.607758 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e462c28-dd50-43f2-883f-4c42b3a32821 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:00.608236 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e462c28-dd50-43f2-883f-4c42b3a32821 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:00.617165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 130/252] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:00.617165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d427dfd-cb5a-49a9-aeba-9413c484ef23 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:00.617165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d427dfd-cb5a-49a9-aeba-9413c484ef23 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:00.617165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d427dfd-cb5a-49a9-aeba-9413c484ef23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:00.652975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-4d427dfd-cb5a-49a9-aeba-9413c484ef23 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:00.655517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505b210-2ae8-447d-9c01-0be5a50600c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:00.656139 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505b210-2ae8-447d-9c01-0be5a50600c3 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:00.656473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505b210-2ae8-447d-9c01-0be5a50600c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:00.656750 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505b210-2ae8-447d-9c01-0be5a50600c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:00.657441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505b210-2ae8-447d-9c01-0be5a50600c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:00.657791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505b210-2ae8-447d-9c01-0be5a50600c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:00.659504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505b210-2ae8-447d-9c01-0be5a50600c3 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:00.660261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-4d427dfd-cb5a-49a9-aeba-9413c484ef23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:00.677820 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 123/253] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:00.683979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:00.685855 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505b210-2ae8-447d-9c01-0be5a50600c3 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:00.797182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 131/254] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:00.826350 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:00.855439 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505b210-2ae8-447d-9c01-0be5a50600c3 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:00.899549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22276d26-393d-4158-8d04-7ad9337f8018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:00.900597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22276d26-393d-4158-8d04-7ad9337f8018 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:00.901089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22276d26-393d-4158-8d04-7ad9337f8018 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:00.901583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22276d26-393d-4158-8d04-7ad9337f8018 None admin] PATH_INFO: `/v3/users/d00c15ca137346aea94f5698900fe15c` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:00.902966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22276d26-393d-4158-8d04-7ad9337f8018 None admin] RBAC: Authorizing `identity:get_user(user_id=d00c15ca137346aea94f5698900fe15c)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:00.904683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22276d26-393d-4158-8d04-7ad9337f8018 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:00.910308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d427dfd-cb5a-49a9-aeba-9413c484ef23 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:00.910497 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 124/255] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:09:00 2022] GET /identity/v3/users/d00c15ca137346aea94f5698900fe15c => generated 283 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:00.918217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22276d26-393d-4158-8d04-7ad9337f8018 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:01.039087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22276d26-393d-4158-8d04-7ad9337f8018 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:01.078252 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a779fb9-3390-469a-8c4b-bc84e4c6ac32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:01.078816 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a779fb9-3390-469a-8c4b-bc84e4c6ac32 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:01.079090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a779fb9-3390-469a-8c4b-bc84e4c6ac32 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:01.079355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a779fb9-3390-469a-8c4b-bc84e4c6ac32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:01.106830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:01.152757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d427dfd-cb5a-49a9-aeba-9413c484ef23 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:01.230811 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a779fb9-3390-469a-8c4b-bc84e4c6ac32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:01.232625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a779fb9-3390-469a-8c4b-bc84e4c6ac32 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:01.241083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74718f79-b179-4841-ba91-d466a87b11b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:01.241780 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74718f79-b179-4841-ba91-d466a87b11b1 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:01.242069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74718f79-b179-4841-ba91-d466a87b11b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:01.242343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74718f79-b179-4841-ba91-d466a87b11b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:01.272866 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74718f79-b179-4841-ba91-d466a87b11b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:01.277929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-74718f79-b179-4841-ba91-d466a87b11b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:01.277929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:01.277929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:01.277929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:01.277929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:01.277929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:01.277929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:01.277929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:01.277929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:01.277929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:01.277929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:01.277929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:01.277929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:01.277929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:01.277929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:01.277929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:01.278835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:01.278835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:01.278835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:01.278835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:01.278835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:01.278835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:01.278835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:01.278835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:01.278835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:01.278835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:01.278835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:01.278835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:01.278835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:01.278835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:01.278835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:01.278835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:01.280017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:01.280017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:01.280017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:01.280017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:01.280017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:01.280017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:01.280017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:01.280017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:01.280017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:01.280017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:01.280017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:01.280017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:01.280017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 132/256] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:01.285341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74718f79-b179-4841-ba91-d466a87b11b1 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:01.303749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:01.339882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 125/257] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:01.340673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a779fb9-3390-469a-8c4b-bc84e4c6ac32 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:01.393858 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:01.454930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a779fb9-3390-469a-8c4b-bc84e4c6ac32 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:01.505231 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e342ef-926f-48ed-9514-03c68fe3c940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:01.506003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e342ef-926f-48ed-9514-03c68fe3c940 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:01.506277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e342ef-926f-48ed-9514-03c68fe3c940 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:01.506546 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e342ef-926f-48ed-9514-03c68fe3c940 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:01.507216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e342ef-926f-48ed-9514-03c68fe3c940 None admin] RBAC: Adding query filter params (name=member) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:01.507516 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e342ef-926f-48ed-9514-03c68fe3c940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:01.508904 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e342ef-926f-48ed-9514-03c68fe3c940 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:01.518036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 126/258] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:09:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:01.519104 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e342ef-926f-48ed-9514-03c68fe3c940 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:01.548323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74718f79-b179-4841-ba91-d466a87b11b1 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:01.618224 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1c078-f8cf-409d-a6fb-116dd031bef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:01.618832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1c078-f8cf-409d-a6fb-116dd031bef5 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:01.619115 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1c078-f8cf-409d-a6fb-116dd031bef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:01.619389 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1c078-f8cf-409d-a6fb-116dd031bef5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:01.629276 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e342ef-926f-48ed-9514-03c68fe3c940 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:01.636508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:01.644381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1c078-f8cf-409d-a6fb-116dd031bef5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:01.649301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-a0c1c078-f8cf-409d-a6fb-116dd031bef5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:09:01.649301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:01.649301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:01.649301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:01.649301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:01.649301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:01.649301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:01.649301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:01.649301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:01.649301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:01.649301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:01.649301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:01.649301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:01.649301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:01.649301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:01.649301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:01.651712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:01.651712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:01.651712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:01.651712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:01.651712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:01.651712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:01.651712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:01.651712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:01.651712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:01.651712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:01.651712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:01.651712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:01.651712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:01.651712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:01.651712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:01.651712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:01.653645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:01.653645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:01.653645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:01.653645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:01.653645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:01.653645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:01.653645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:01.653645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:01.653645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:01.653645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:01.653645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:09:01.653645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:01.653645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 133/259] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:09:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:01.653645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1c078-f8cf-409d-a6fb-116dd031bef5 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:01.677094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6948a-9abd-4269-8e8e-9f479ef32f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:01.677762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6948a-9abd-4269-8e8e-9f479ef32f62 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:01.678040 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6948a-9abd-4269-8e8e-9f479ef32f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:01.678310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6948a-9abd-4269-8e8e-9f479ef32f62 None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:01.680257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd6948a-9abd-4269-8e8e-9f479ef32f62 None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:01.681734 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd6948a-9abd-4269-8e8e-9f479ef32f62 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:01.684552 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 127/260] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:01 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:01.690428 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6948a-9abd-4269-8e8e-9f479ef32f62 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:01.780941 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:01.850528 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1c078-f8cf-409d-a6fb-116dd031bef5 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:01.894135 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6948a-9abd-4269-8e8e-9f479ef32f62 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:01.905392 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58fc020-7170-452f-b7cf-a1d94a4f57f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:01.906550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fc020-7170-452f-b7cf-a1d94a4f57f0 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:01.906835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fc020-7170-452f-b7cf-a1d94a4f57f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:01.907115 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58fc020-7170-452f-b7cf-a1d94a4f57f0 None admin] PATH_INFO: `/v3/users/0555244f65384fd5a9d967f604e5d328` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:01.908330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58fc020-7170-452f-b7cf-a1d94a4f57f0 None admin] RBAC: Authorizing `identity:get_user(user_id=0555244f65384fd5a9d967f604e5d328)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:01.910388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58fc020-7170-452f-b7cf-a1d94a4f57f0 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:01.912908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 134/261] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:09:01 2022] GET /identity/v3/users/0555244f65384fd5a9d967f604e5d328 => generated 297 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:01.913763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58fc020-7170-452f-b7cf-a1d94a4f57f0 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:01.960471 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fefcab-fc3e-4f3b-8e9a-2b99ceb04d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:01.961089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fefcab-fc3e-4f3b-8e9a-2b99ceb04d0a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:01.961394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fefcab-fc3e-4f3b-8e9a-2b99ceb04d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:01.961664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fefcab-fc3e-4f3b-8e9a-2b99ceb04d0a None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:01.962450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fefcab-fc3e-4f3b-8e9a-2b99ceb04d0a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:01.962748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fefcab-fc3e-4f3b-8e9a-2b99ceb04d0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:01.964230 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fefcab-fc3e-4f3b-8e9a-2b99ceb04d0a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:01.976190 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 128/262] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:01.977007 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fefcab-fc3e-4f3b-8e9a-2b99ceb04d0a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:02.046420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:02.053509 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58fc020-7170-452f-b7cf-a1d94a4f57f0 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:02.118955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c07d9-229d-42dd-9a6b-80d17ffdb5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:02.119876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c07d9-229d-42dd-9a6b-80d17ffdb5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:02.119984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fefcab-fc3e-4f3b-8e9a-2b99ceb04d0a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:02.120473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c07d9-229d-42dd-9a6b-80d17ffdb5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:02.120950 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c07d9-229d-42dd-9a6b-80d17ffdb5c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:02.121874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c07d9-229d-42dd-9a6b-80d17ffdb5c3 None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=81a7733885564252af45b01ea11bfed2, user.id=d00c15ca137346aea94f5698900fe15c) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:02.122361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c07d9-229d-42dd-9a6b-80d17ffdb5c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:02.129166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c07d9-229d-42dd-9a6b-80d17ffdb5c3 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:02.141424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:02.150953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 135/263] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:09:01 2022] GET /identity/v3/role_assignments?user.id=d00c15ca137346aea94f5698900fe15c&scope.project.id=81a7733885564252af45b01ea11bfed2&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:02.151801 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c07d9-229d-42dd-9a6b-80d17ffdb5c3 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:02.171282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8078f8c-f44c-4cf9-b668-799526738e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:02.172018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8078f8c-f44c-4cf9-b668-799526738e3e None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:02.172373 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8078f8c-f44c-4cf9-b668-799526738e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:02.172718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8078f8c-f44c-4cf9-b668-799526738e3e None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:02.178156 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8078f8c-f44c-4cf9-b668-799526738e3e None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:02.185010 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8078f8c-f44c-4cf9-b668-799526738e3e None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:02.193684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 129/264] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:01 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:02.199656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8078f8c-f44c-4cf9-b668-799526738e3e None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:02.364765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c07d9-229d-42dd-9a6b-80d17ffdb5c3 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:02.403833 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8078f8c-f44c-4cf9-b668-799526738e3e None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:02.416737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:02.443377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f079be-57ab-40b1-a81b-a7990aa69958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:02.444037 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f079be-57ab-40b1-a81b-a7990aa69958 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:02.444326 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f079be-57ab-40b1-a81b-a7990aa69958 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:02.444609 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f079be-57ab-40b1-a81b-a7990aa69958 None admin] PATH_INFO: `/v3/users/d00c15ca137346aea94f5698900fe15c` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:02.445793 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f079be-57ab-40b1-a81b-a7990aa69958 None admin] RBAC: Authorizing `identity:get_user(user_id=d00c15ca137346aea94f5698900fe15c)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:02.447267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f079be-57ab-40b1-a81b-a7990aa69958 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:02.455472 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 136/265] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:09:02 2022] GET /identity/v3/users/d00c15ca137346aea94f5698900fe15c => generated 283 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:02.461100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f079be-57ab-40b1-a81b-a7990aa69958 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:02.485517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49c709-1646-4d34-acf7-a98ff6371ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:02.486534 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49c709-1646-4d34-acf7-a98ff6371ec1 None admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:02.486875 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49c709-1646-4d34-acf7-a98ff6371ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:02.487360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e49c709-1646-4d34-acf7-a98ff6371ec1 None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2/users/0555244f65384fd5a9d967f604e5d328/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:02.494678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49c709-1646-4d34-acf7-a98ff6371ec1 None admin] RBAC: Authorizing `identity:create_grant(project_id=81a7733885564252af45b01ea11bfed2, user_id=0555244f65384fd5a9d967f604e5d328, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:02.496464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e49c709-1646-4d34-acf7-a98ff6371ec1 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:02.513209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 130/266] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:09:02 2022] PUT /identity/v3/projects/81a7733885564252af45b01ea11bfed2/users/0555244f65384fd5a9d967f604e5d328/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:09:02.583215 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:02.627370 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f079be-57ab-40b1-a81b-a7990aa69958 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:02.672075 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa836b-5e6c-46fd-94d9-6cb37a7f1d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:02.672631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa836b-5e6c-46fd-94d9-6cb37a7f1d02 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:02.672915 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa836b-5e6c-46fd-94d9-6cb37a7f1d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:02.673288 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa836b-5e6c-46fd-94d9-6cb37a7f1d02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:02.762524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa836b-5e6c-46fd-94d9-6cb37a7f1d02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:02.764008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa836b-5e6c-46fd-94d9-6cb37a7f1d02 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:02.848490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:02.901113 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 137/267] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 12:09:02.906454 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49c709-1646-4d34-acf7-a98ff6371ec1 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:03.017227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e49c709-1646-4d34-acf7-a98ff6371ec1 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:03.059951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990e4349-664f-4a54-ae31-a8d8b8928363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:03.060604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e4349-664f-4a54-ae31-a8d8b8928363 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:03.060884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e4349-664f-4a54-ae31-a8d8b8928363 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:03.061229 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990e4349-664f-4a54-ae31-a8d8b8928363 None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:03.062404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e4349-664f-4a54-ae31-a8d8b8928363 None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:03.063843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990e4349-664f-4a54-ae31-a8d8b8928363 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:03.066085 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 131/268] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:02 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:03.071871 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa836b-5e6c-46fd-94d9-6cb37a7f1d02 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:03.170764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b800f9b-68ec-4199-8e35-7b1a709acbb7 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:03.170764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b800f9b-68ec-4199-8e35-7b1a709acbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:03.171093 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b800f9b-68ec-4199-8e35-7b1a709acbb7 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:03.172671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 132/269] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:03.178984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c096d55-9faa-4657-9255-b0eee2925bdf None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:03.179300 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c096d55-9faa-4657-9255-b0eee2925bdf None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:03.179531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c096d55-9faa-4657-9255-b0eee2925bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:03.213919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-0c096d55-9faa-4657-9255-b0eee2925bdf None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:03.217920 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-0c096d55-9faa-4657-9255-b0eee2925bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:03.264943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa836b-5e6c-46fd-94d9-6cb37a7f1d02 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:03.275345 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:03.308184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342c236-cb9d-4b6f-952b-e89b2c0f01ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:03.308788 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342c236-cb9d-4b6f-952b-e89b2c0f01ca None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:03.309066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342c236-cb9d-4b6f-952b-e89b2c0f01ca None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:03.309364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342c236-cb9d-4b6f-952b-e89b2c0f01ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:03.310086 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342c236-cb9d-4b6f-952b-e89b2c0f01ca None admin] RBAC: Adding query filter params (role.id=8e9576d16f194938b86a1d1e087895bd, scope.project.id=81a7733885564252af45b01ea11bfed2, user.id=d00c15ca137346aea94f5698900fe15c) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:03.310391 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342c236-cb9d-4b6f-952b-e89b2c0f01ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:03.311833 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342c236-cb9d-4b6f-952b-e89b2c0f01ca None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:03.318300 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 138/270] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:09:03 2022] GET /identity/v3/role_assignments?user.id=d00c15ca137346aea94f5698900fe15c&scope.project.id=81a7733885564252af45b01ea11bfed2&role.id=8e9576d16f194938b86a1d1e087895bd => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:03.319312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea4af6-5c75-4a7e-884c-bf32200de532 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:03.319593 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea4af6-5c75-4a7e-884c-bf32200de532 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:03.319863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea4af6-5c75-4a7e-884c-bf32200de532 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:03.321273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 139/271] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:03.322089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc520fd1-b8e4-4787-a6ec-2eebcb1f21e2 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:03.322362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc520fd1-b8e4-4787-a6ec-2eebcb1f21e2 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:03.322635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc520fd1-b8e4-4787-a6ec-2eebcb1f21e2 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:03.324041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 140/272] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:03.329195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 133/273] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:03.330657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaab7b7-d327-4d08-8764-c54ce83d6787 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:03.330949 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaab7b7-d327-4d08-8764-c54ce83d6787 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:03.331222 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaab7b7-d327-4d08-8764-c54ce83d6787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:03.334309 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3af51-969a-4b3b-a3f4-ec7af7fff288 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:03.334597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3af51-969a-4b3b-a3f4-ec7af7fff288 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:03.334871 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d3af51-969a-4b3b-a3f4-ec7af7fff288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:03.355192 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-0eaab7b7-d327-4d08-8764-c54ce83d6787 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:03.355632 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-29d3af51-969a-4b3b-a3f4-ec7af7fff288 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:03.356847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-29d3af51-969a-4b3b-a3f4-ec7af7fff288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:03.357194 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-0eaab7b7-d327-4d08-8764-c54ce83d6787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:03.386711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:03.468690 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 134/274] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:03.470189 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 141/275] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:03.653064 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaab7b7-d327-4d08-8764-c54ce83d6787 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:03.750266 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:03.763700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaab7b7-d327-4d08-8764-c54ce83d6787 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:03.779762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3af51-969a-4b3b-a3f4-ec7af7fff288 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:03.802892 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97568e35-5e29-4df0-8546-89eef6bf4d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:03.803563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97568e35-5e29-4df0-8546-89eef6bf4d58 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:03.803847 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97568e35-5e29-4df0-8546-89eef6bf4d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:03.804120 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97568e35-5e29-4df0-8546-89eef6bf4d58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:03.805579 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97568e35-5e29-4df0-8546-89eef6bf4d58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:03.807017 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97568e35-5e29-4df0-8546-89eef6bf4d58 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:03.808755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 135/276] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:03.809393 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97568e35-5e29-4df0-8546-89eef6bf4d58 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:03.913583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d3af51-969a-4b3b-a3f4-ec7af7fff288 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:03.918900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97568e35-5e29-4df0-8546-89eef6bf4d58 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:03.953811 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd363de4-e788-4cc0-8040-5f774fd0d2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:03.954671 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363de4-e788-4cc0-8040-5f774fd0d2eb None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:03.954900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc3cea-0160-4efe-951a-97bf1ed16541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:03.955316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363de4-e788-4cc0-8040-5f774fd0d2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:03.955562 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc3cea-0160-4efe-951a-97bf1ed16541 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:03.955860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc3cea-0160-4efe-951a-97bf1ed16541 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:03.956111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc3cea-0160-4efe-951a-97bf1ed16541 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:03.956307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363de4-e788-4cc0-8040-5f774fd0d2eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:03.966529 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd363de4-e788-4cc0-8040-5f774fd0d2eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:03.966845 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc3cea-0160-4efe-951a-97bf1ed16541 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:03.972217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-45fc3cea-0160-4efe-951a-97bf1ed16541 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:03.972217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:03.972217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:03.972217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:03.972217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:03.972217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:03.972217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:03.972217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:03.972217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:03.972217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:03.972217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:03.972217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:03.972217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:03.972217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:03.972217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:03.972217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:03.973278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:03.973278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:03.973278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:03.973278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:03.973278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:03.973278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:03.973278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:03.973278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:03.973278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:03.973278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:03.973278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:03.973278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:03.973278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:03.973278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:03.973278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:03.973278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:03.974157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:03.974157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:03.974157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:03.974157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:03.974157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:03.974157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:03.974157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:03.974157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:03.974157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:03.974157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:03.974157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:03.974157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:03.974157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 136/277] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:03.974871 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc3cea-0160-4efe-951a-97bf1ed16541 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:03.975254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-fd363de4-e788-4cc0-8040-5f774fd0d2eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:09:03.975254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:03.975254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:03.975254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:03.975254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:03.975254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:03.975254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:03.975254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:03.975254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:03.975254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:03.975254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:03.975254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:03.975254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:03.975254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:03.975254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:03.975254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:03.976119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:03.976119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:03.976119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:03.976119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:03.976119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:03.976119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:03.976119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:03.976119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:03.976119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:03.976119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:03.976119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:03.976119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:03.976119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:03.976119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:03.976119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:03.976119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:03.977980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:03.977980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:03.977980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:03.977980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:03.977980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:03.977980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:03.977980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:03.977980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:03.977980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:03.977980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:03.977980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:09:03.977980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:03.977980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 142/278] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:03.979466 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd363de4-e788-4cc0-8040-5f774fd0d2eb None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:04.004880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:04.009544 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:04.087715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc3cea-0160-4efe-951a-97bf1ed16541 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:04.092496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd363de4-e788-4cc0-8040-5f774fd0d2eb None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:04.121877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c8f49-77e2-403a-81bc-6964881ef89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:04.122799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c8f49-77e2-403a-81bc-6964881ef89b None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:04.123217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c8f49-77e2-403a-81bc-6964881ef89b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:04.123487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c8f49-77e2-403a-81bc-6964881ef89b None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:04.124195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c8f49-77e2-403a-81bc-6964881ef89b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:04.125578 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c8f49-77e2-403a-81bc-6964881ef89b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:04.129830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c1f1d-6504-4787-af3c-2e5a5d94bdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:04.130434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c1f1d-6504-4787-af3c-2e5a5d94bdd2 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:04.130715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c1f1d-6504-4787-af3c-2e5a5d94bdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:04.131023 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c1f1d-6504-4787-af3c-2e5a5d94bdd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:04.131647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c1f1d-6504-4787-af3c-2e5a5d94bdd2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:04.131950 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c1f1d-6504-4787-af3c-2e5a5d94bdd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:04.133366 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c1f1d-6504-4787-af3c-2e5a5d94bdd2 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:04.149900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 143/279] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:09:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:04.149900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c1f1d-6504-4787-af3c-2e5a5d94bdd2 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:04.157214 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 137/280] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:09:03 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:09:04.158111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c8f49-77e2-403a-81bc-6964881ef89b None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:04.259226 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:04.280352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c1f1d-6504-4787-af3c-2e5a5d94bdd2 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:04.284601 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:04.312783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c8f49-77e2-403a-81bc-6964881ef89b None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:04.322328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a550a58-32c5-463b-9f0c-314903a11036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:04.322946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a550a58-32c5-463b-9f0c-314903a11036 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:04.323389 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a550a58-32c5-463b-9f0c-314903a11036 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:04.323800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a550a58-32c5-463b-9f0c-314903a11036 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:04.324622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a550a58-32c5-463b-9f0c-314903a11036 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:04.325202 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a550a58-32c5-463b-9f0c-314903a11036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:04.326828 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a550a58-32c5-463b-9f0c-314903a11036 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:04.336964 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 144/281] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:09:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:04.342681 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a550a58-32c5-463b-9f0c-314903a11036 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:04.360145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488dc0f-6254-4e5f-9d8a-8370d64a8cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:04.361401 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488dc0f-6254-4e5f-9d8a-8370d64a8cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:04.362149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488dc0f-6254-4e5f-9d8a-8370d64a8cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:04.362646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488dc0f-6254-4e5f-9d8a-8370d64a8cc6 None admin] PATH_INFO: `/v3/users/4ea040edfa734126823bd4b21e953ac4` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:04.377902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488dc0f-6254-4e5f-9d8a-8370d64a8cc6 None admin] RBAC: Authorizing `identity:get_user(user_id=4ea040edfa734126823bd4b21e953ac4)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:04.379700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488dc0f-6254-4e5f-9d8a-8370d64a8cc6 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:04.381519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 138/282] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:09:04 2022] GET /identity/v3/users/4ea040edfa734126823bd4b21e953ac4 => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:04.387366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488dc0f-6254-4e5f-9d8a-8370d64a8cc6 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:04.478124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a550a58-32c5-463b-9f0c-314903a11036 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:04.522947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd4c2f-e219-450b-bb64-742ff5b10ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:04.523586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd4c2f-e219-450b-bb64-742ff5b10ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:04.523863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd4c2f-e219-450b-bb64-742ff5b10ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:04.524135 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd4c2f-e219-450b-bb64-742ff5b10ed5 None admin] PATH_INFO: `/v3/users/c0c2f447c91b455db5c8e72b7497f907` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:04.525777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488dc0f-6254-4e5f-9d8a-8370d64a8cc6 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:04.528976 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd4c2f-e219-450b-bb64-742ff5b10ed5 None admin] RBAC: Authorizing `identity:get_user(user_id=c0c2f447c91b455db5c8e72b7497f907)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:04.530366 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd4c2f-e219-450b-bb64-742ff5b10ed5 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:04.531981 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 145/283] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:09:04 2022] GET /identity/v3/users/c0c2f447c91b455db5c8e72b7497f907 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:04.540142 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd4c2f-e219-450b-bb64-742ff5b10ed5 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:04.558642 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:04.576261 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043583b5-60e3-4b27-909a-bd6d3913b335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:04.577474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043583b5-60e3-4b27-909a-bd6d3913b335 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:04.577829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043583b5-60e3-4b27-909a-bd6d3913b335 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:04.578427 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043583b5-60e3-4b27-909a-bd6d3913b335 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:04.579231 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043583b5-60e3-4b27-909a-bd6d3913b335 None admin] RBAC: Adding query filter params (role.id=2201e005d5ef449896a0e68a940678ec, user.id=4ea040edfa734126823bd4b21e953ac4, scope.system=all) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:04.579617 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043583b5-60e3-4b27-909a-bd6d3913b335 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:04.582055 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043583b5-60e3-4b27-909a-bd6d3913b335 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:04.587098 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:04.589121 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 139/284] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Sep 28 12:09:04 2022] GET /identity/v3/role_assignments?user.id=4ea040edfa734126823bd4b21e953ac4&scope.system=all&role.id=2201e005d5ef449896a0e68a940678ec => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:04.696103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd4c2f-e219-450b-bb64-742ff5b10ed5 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:04.746148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d04e47-ca86-4847-8b57-3f09f5e7b8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:04.746736 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d04e47-ca86-4847-8b57-3f09f5e7b8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:04.747020 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d04e47-ca86-4847-8b57-3f09f5e7b8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:04.747294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d04e47-ca86-4847-8b57-3f09f5e7b8b2 None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:04.748604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d04e47-ca86-4847-8b57-3f09f5e7b8b2 None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:04.750119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d04e47-ca86-4847-8b57-3f09f5e7b8b2 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:04.751863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 146/285] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:04 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:04.758790 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043583b5-60e3-4b27-909a-bd6d3913b335 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:04.884212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043583b5-60e3-4b27-909a-bd6d3913b335 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:04.929929 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2702f5-e198-486b-bd65-9994acc72e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:04.930689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2702f5-e198-486b-bd65-9994acc72e86 None admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:04.930877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2702f5-e198-486b-bd65-9994acc72e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:04.931150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2702f5-e198-486b-bd65-9994acc72e86 None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2/users/c0c2f447c91b455db5c8e72b7497f907/roles/edf27bedabd24681b24ec2a72a774a9c` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:04.934917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2702f5-e198-486b-bd65-9994acc72e86 None admin] RBAC: Authorizing `identity:create_grant(project_id=81a7733885564252af45b01ea11bfed2, user_id=c0c2f447c91b455db5c8e72b7497f907, role_id=edf27bedabd24681b24ec2a72a774a9c)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:04.936363 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2702f5-e198-486b-bd65-9994acc72e86 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:04.963422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 140/286] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:09:04 2022] PUT /identity/v3/projects/81a7733885564252af45b01ea11bfed2/users/c0c2f447c91b455db5c8e72b7497f907/roles/edf27bedabd24681b24ec2a72a774a9c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:09:05.085046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f43f1e-35de-47d4-aa73-fe62f6a6230e None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:05.085564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f43f1e-35de-47d4-aa73-fe62f6a6230e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:05.085564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f43f1e-35de-47d4-aa73-fe62f6a6230e None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:05.087244 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 147/287] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:05.092731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a1f71-d04c-498b-8cd1-d604d873e2df None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:05.093023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a1f71-d04c-498b-8cd1-d604d873e2df None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:05.093299 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a1f71-d04c-498b-8cd1-d604d873e2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:05.115305 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-cd3a1f71-d04c-498b-8cd1-d604d873e2df None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:05.116609 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-cd3a1f71-d04c-498b-8cd1-d604d873e2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:05.179291 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:05.198356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 141/288] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:05.480301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f43f1e-35de-47d4-aa73-fe62f6a6230e None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:05.587837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:05.610243 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f43f1e-35de-47d4-aa73-fe62f6a6230e None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:05.656319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5ebda-e6f7-4c54-8746-3fd8f7ba40ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:05.657008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5ebda-e6f7-4c54-8746-3fd8f7ba40ea None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:05.657312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5ebda-e6f7-4c54-8746-3fd8f7ba40ea None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:05.657582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5ebda-e6f7-4c54-8746-3fd8f7ba40ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:05.671450 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd5ebda-e6f7-4c54-8746-3fd8f7ba40ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:05.676557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-fcd5ebda-e6f7-4c54-8746-3fd8f7ba40ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:05.676557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:05.676557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:05.676557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:05.676557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:05.676557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:05.676557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:05.676557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:05.676557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:05.676557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:05.676557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:05.676557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:05.676557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:05.676557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:05.676557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:05.676557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:05.677475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:05.677475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:05.677475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:05.677475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:05.677475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:05.677475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:05.677475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:05.677475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:05.677475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:05.677475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:05.677475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:05.677475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:05.677475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:05.677475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:05.677475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:05.677475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:05.678667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:05.678667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:05.678667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:05.678667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:05.678667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:05.678667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:05.678667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:05.678667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:05.678667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:05.678667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:05.678667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:05.678667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:05.678667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 148/289] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:05.683962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a1f71-d04c-498b-8cd1-d604d873e2df None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:05.803315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a1f71-d04c-498b-8cd1-d604d873e2df None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:05.845900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f5a68-7176-4a49-9b1e-84d4374345c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:05.846574 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f5a68-7176-4a49-9b1e-84d4374345c5 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:05.846855 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f5a68-7176-4a49-9b1e-84d4374345c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:05.847162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f5a68-7176-4a49-9b1e-84d4374345c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:05.847824 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f5a68-7176-4a49-9b1e-84d4374345c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:05.848123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f5a68-7176-4a49-9b1e-84d4374345c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:05.849509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f5a68-7176-4a49-9b1e-84d4374345c5 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:05.861848 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 142/290] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:09:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:05.867592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5ebda-e6f7-4c54-8746-3fd8f7ba40ea None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:06.017664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5ebda-e6f7-4c54-8746-3fd8f7ba40ea None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:06.067706 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a126b-b5e2-4e55-a4b8-8df4d9e18c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:06.068672 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a126b-b5e2-4e55-a4b8-8df4d9e18c28 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:06.069208 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a126b-b5e2-4e55-a4b8-8df4d9e18c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:06.069698 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a126b-b5e2-4e55-a4b8-8df4d9e18c28 None admin] PATH_INFO: `/v3/users/0555244f65384fd5a9d967f604e5d328` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:06.071099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a126b-b5e2-4e55-a4b8-8df4d9e18c28 None admin] RBAC: Authorizing `identity:get_user(user_id=0555244f65384fd5a9d967f604e5d328)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:06.072756 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a126b-b5e2-4e55-a4b8-8df4d9e18c28 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:06.075520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 149/291] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:09:05 2022] GET /identity/v3/users/0555244f65384fd5a9d967f604e5d328 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:06.080180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f5a68-7176-4a49-9b1e-84d4374345c5 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:06.179786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:06.232563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f5a68-7176-4a49-9b1e-84d4374345c5 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:06.273638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15098f99-d44e-4b05-bf47-2c238da929e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:06.274247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15098f99-d44e-4b05-bf47-2c238da929e3 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:06.274524 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15098f99-d44e-4b05-bf47-2c238da929e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:06.274793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15098f99-d44e-4b05-bf47-2c238da929e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:06.365374 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15098f99-d44e-4b05-bf47-2c238da929e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:06.366833 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15098f99-d44e-4b05-bf47-2c238da929e3 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:06.431006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:06.498573 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 143/292] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:06.504207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a126b-b5e2-4e55-a4b8-8df4d9e18c28 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:06.529007 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:06.644750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a126b-b5e2-4e55-a4b8-8df4d9e18c28 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:06.690408 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e971a-cf8e-4263-9eb9-7ec63fafb5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:06.691043 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e971a-cf8e-4263-9eb9-7ec63fafb5fd None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:06.691318 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e971a-cf8e-4263-9eb9-7ec63fafb5fd None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:06.691586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e971a-cf8e-4263-9eb9-7ec63fafb5fd None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:06.692763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e971a-cf8e-4263-9eb9-7ec63fafb5fd None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:06.694214 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e971a-cf8e-4263-9eb9-7ec63fafb5fd None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:06.696041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 150/293] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:06 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:06.701665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15098f99-d44e-4b05-bf47-2c238da929e3 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:06.853041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15098f99-d44e-4b05-bf47-2c238da929e3 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:06.900572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f2fda6-d137-42bc-8013-862d0b2799e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:06.901116 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2fda6-d137-42bc-8013-862d0b2799e3 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:06.901417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2fda6-d137-42bc-8013-862d0b2799e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:06.901685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2fda6-d137-42bc-8013-862d0b2799e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:06.902406 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f2fda6-d137-42bc-8013-862d0b2799e3 None admin] RBAC: Adding query filter params (role.id=2201e005d5ef449896a0e68a940678ec, scope.project.id=81a7733885564252af45b01ea11bfed2, user.id=0555244f65384fd5a9d967f604e5d328) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:06.902708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f2fda6-d137-42bc-8013-862d0b2799e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:06.904123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f2fda6-d137-42bc-8013-862d0b2799e3 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:06.911352 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 144/294] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:09:06 2022] GET /identity/v3/role_assignments?user.id=0555244f65384fd5a9d967f604e5d328&scope.project.id=81a7733885564252af45b01ea11bfed2&role.id=2201e005d5ef449896a0e68a940678ec => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:06.932235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13ac15-711e-49b1-a57c-f67cb681061a None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:06.932545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13ac15-711e-49b1-a57c-f67cb681061a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:06.932783 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13ac15-711e-49b1-a57c-f67cb681061a None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:06.934528 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 151/295] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:06.940611 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c54203-d819-4dbe-aa5a-df61d867dd62 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:06.940914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c54203-d819-4dbe-aa5a-df61d867dd62 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:06.941183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c54203-d819-4dbe-aa5a-df61d867dd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:06.970432 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-87c54203-d819-4dbe-aa5a-df61d867dd62 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:06.972696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-87c54203-d819-4dbe-aa5a-df61d867dd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:06.991762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:07.073984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 145/296] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:07.130511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0f55b-8420-44a2-8307-f8b4a50934ff None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:07.131117 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0f55b-8420-44a2-8307-f8b4a50934ff None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:07.131599 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0f55b-8420-44a2-8307-f8b4a50934ff None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:07.133571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 152/297] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:07.138782 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69c92c-8a2b-41b7-b104-55ec9084fec4 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:07.139255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69c92c-8a2b-41b7-b104-55ec9084fec4 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:07.139339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69c92c-8a2b-41b7-b104-55ec9084fec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:07.162351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-fe69c92c-8a2b-41b7-b104-55ec9084fec4 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:07.163472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-fe69c92c-8a2b-41b7-b104-55ec9084fec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:07.234892 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 146/298] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:07.410975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0f55b-8420-44a2-8307-f8b4a50934ff None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:07.510323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:07.534818 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0f55b-8420-44a2-8307-f8b4a50934ff None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:07.536006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781e30b-3197-4218-9298-9d5cfb0b6c4b None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:07.536523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781e30b-3197-4218-9298-9d5cfb0b6c4b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:07.537016 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781e30b-3197-4218-9298-9d5cfb0b6c4b None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:07.538809 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 147/299] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:09:07.543970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c577762-f3e4-42bb-af96-57e7dea0b0d7 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:07.544508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c577762-f3e4-42bb-af96-57e7dea0b0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:07.544846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c577762-f3e4-42bb-af96-57e7dea0b0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:07.564143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-0c577762-f3e4-42bb-af96-57e7dea0b0d7 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:07.566104 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-0c577762-f3e4-42bb-af96-57e7dea0b0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:07.571960 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00faa76-0649-4e1b-920a-4d23b6e5c98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:07.572596 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00faa76-0649-4e1b-920a-4d23b6e5c98f None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:07.572878 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00faa76-0649-4e1b-920a-4d23b6e5c98f None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:07.573179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00faa76-0649-4e1b-920a-4d23b6e5c98f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:07.586959 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00faa76-0649-4e1b-920a-4d23b6e5c98f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:07.594695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-f00faa76-0649-4e1b-920a-4d23b6e5c98f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:09:07.594695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:07.594695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:07.594695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:07.594695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:07.594695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:07.594695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:07.594695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:07.594695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:07.594695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:07.594695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:07.594695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:07.594695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:07.594695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:07.594695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:07.594695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:07.595587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:07.595587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:07.595587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:07.595587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:07.595587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:07.595587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:07.595587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:07.595587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:07.595587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:07.595587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:07.595587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:07.595587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:07.595587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:07.595587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:07.595587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:07.595587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:07.596397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:07.596397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:07.596397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:07.596397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:07.596397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:07.596397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:07.596397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:07.596397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:07.596397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:07.596397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:07.596397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:09:07.596397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:07.596397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 153/300] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:07.601439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00faa76-0649-4e1b-920a-4d23b6e5c98f None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:07.639090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 148/301] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:09:07.640159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c577762-f3e4-42bb-af96-57e7dea0b0d7 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:07.644792 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:07.742040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00faa76-0649-4e1b-920a-4d23b6e5c98f None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:07.751516 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c577762-f3e4-42bb-af96-57e7dea0b0d7 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:07.777399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8fc54-f673-4f2d-9e11-df75613c6cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:07.777952 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8fc54-f673-4f2d-9e11-df75613c6cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:07.778228 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8fc54-f673-4f2d-9e11-df75613c6cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:07.778554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8fc54-f673-4f2d-9e11-df75613c6cf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:07.779705 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8fc54-f673-4f2d-9e11-df75613c6cf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:07.781487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8fc54-f673-4f2d-9e11-df75613c6cf0 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:07.783765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 154/302] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:07.790302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8fc54-f673-4f2d-9e11-df75613c6cf0 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:07.801263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fdf55-f8d0-49e6-ad58-26104fa72112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:07.802063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fdf55-f8d0-49e6-ad58-26104fa72112 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:07.802469 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fdf55-f8d0-49e6-ad58-26104fa72112 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:07.802832 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225fdf55-f8d0-49e6-ad58-26104fa72112 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:07.803325 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:07.803670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fdf55-f8d0-49e6-ad58-26104fa72112 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:07.804232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fdf55-f8d0-49e6-ad58-26104fa72112 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:07.805718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225fdf55-f8d0-49e6-ad58-26104fa72112 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:07.819973 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 149/303] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:09:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:07.821310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fdf55-f8d0-49e6-ad58-26104fa72112 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:07.897861 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:07.936551 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225fdf55-f8d0-49e6-ad58-26104fa72112 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:07.938707 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8fc54-f673-4f2d-9e11-df75613c6cf0 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:07.973530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94f9c9-ed02-45f1-843b-0af7857a6069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:07.973733 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3778a3-2ddf-49cd-9948-6654928595bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:07.974165 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94f9c9-ed02-45f1-843b-0af7857a6069 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:07.974265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3778a3-2ddf-49cd-9948-6654928595bb None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:07.974437 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94f9c9-ed02-45f1-843b-0af7857a6069 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:07.974540 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3778a3-2ddf-49cd-9948-6654928595bb None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:07.974710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94f9c9-ed02-45f1-843b-0af7857a6069 None admin] PATH_INFO: `/v3/users/31ff84c2f79241c09d73f2abddaadbcc` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:07.974803 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3778a3-2ddf-49cd-9948-6654928595bb None admin] PATH_INFO: `/v3/groups` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:07.975371 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3778a3-2ddf-49cd-9948-6654928595bb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:07.976786 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3778a3-2ddf-49cd-9948-6654928595bb None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:07.989829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94f9c9-ed02-45f1-843b-0af7857a6069 None admin] RBAC: Authorizing `identity:get_user(user_id=31ff84c2f79241c09d73f2abddaadbcc)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:07.991309 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94f9c9-ed02-45f1-843b-0af7857a6069 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:07.993077 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 150/304] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:09:07 2022] GET /identity/v3/users/31ff84c2f79241c09d73f2abddaadbcc => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:07.993741 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94f9c9-ed02-45f1-843b-0af7857a6069 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:07.994750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 155/305] 10.222.0.60 () {60 vars in 1134 bytes} [Wed Sep 28 12:09:07 2022] POST /identity/v3/groups => generated 229 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:09:07.998625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d297a0-2c98-492c-a2bf-71ddbd7c0415 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:07.998911 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d297a0-2c98-492c-a2bf-71ddbd7c0415 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:07.999186 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d297a0-2c98-492c-a2bf-71ddbd7c0415 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:08.000855 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 156/306] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:08.002702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d297a0-2c98-492c-a2bf-71ddbd7c0415 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:08.082707 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:08.109241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94f9c9-ed02-45f1-843b-0af7857a6069 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:08.128671 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d297a0-2c98-492c-a2bf-71ddbd7c0415 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:08.139681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:08.154005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a174e8-bf17-4de3-bab2-3c24fbb33244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:08.154653 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a174e8-bf17-4de3-bab2-3c24fbb33244 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:08.154927 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a174e8-bf17-4de3-bab2-3c24fbb33244 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:08.155196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a174e8-bf17-4de3-bab2-3c24fbb33244 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:08.168407 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a174e8-bf17-4de3-bab2-3c24fbb33244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:08.171613 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724cdd1-f6c1-401d-8d34-091cbb869dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:08.172286 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724cdd1-f6c1-401d-8d34-091cbb869dac None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:08.172491 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724cdd1-f6c1-401d-8d34-091cbb869dac None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:08.172759 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724cdd1-f6c1-401d-8d34-091cbb869dac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:08.173524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724cdd1-f6c1-401d-8d34-091cbb869dac None admin] RBAC: Adding query filter params (role.id=edf27bedabd24681b24ec2a72a774a9c, user.id=31ff84c2f79241c09d73f2abddaadbcc, scope.system=all) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:08.174010 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724cdd1-f6c1-401d-8d34-091cbb869dac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:08.175063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-71a174e8-bf17-4de3-bab2-3c24fbb33244 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:08.175063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:08.175063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:08.175063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:08.175063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:08.175063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:08.175063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:08.175063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:08.175063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:08.175063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:08.175063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:08.175063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:08.175063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:08.175063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:08.175063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:08.175063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:08.175942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:08.175942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:08.175942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:08.175942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:08.175942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:08.175942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:08.175942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:08.175942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:08.175942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:08.175942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:08.175942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:08.175942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:08.175942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:08.175942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:08.175942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:08.175942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:08.176749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:08.176749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:08.176749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:08.176749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:08.176749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:08.176749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:08.176749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:08.176749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:08.176749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:08.176749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:08.176749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:08.176749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:08.177492 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724cdd1-f6c1-401d-8d34-091cbb869dac None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:08.178287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 151/307] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:08.180640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f8018-37c3-49af-8c43-0a5de2f26dd1 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:08.180640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f8018-37c3-49af-8c43-0a5de2f26dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:08.180895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f8018-37c3-49af-8c43-0a5de2f26dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:08.193161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 157/308] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Sep 28 12:09:08 2022] GET /identity/v3/role_assignments?user.id=31ff84c2f79241c09d73f2abddaadbcc&scope.system=all&role.id=edf27bedabd24681b24ec2a72a774a9c => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:08.193161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724cdd1-f6c1-401d-8d34-091cbb869dac None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:08.208415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-6b1f8018-37c3-49af-8c43-0a5de2f26dd1 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:08.209799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-6b1f8018-37c3-49af-8c43-0a5de2f26dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:08.304151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 152/309] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:08.320422 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724cdd1-f6c1-401d-8d34-091cbb869dac None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:08.352809 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:08.360663 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744591c-6a39-4fbd-a89a-b623f085a6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:08.361606 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744591c-6a39-4fbd-a89a-b623f085a6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:08.361883 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744591c-6a39-4fbd-a89a-b623f085a6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:08.362153 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744591c-6a39-4fbd-a89a-b623f085a6f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:08.362910 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744591c-6a39-4fbd-a89a-b623f085a6f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:08.363283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744591c-6a39-4fbd-a89a-b623f085a6f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:08.364766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744591c-6a39-4fbd-a89a-b623f085a6f0 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:08.378269 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 158/310] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:09:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:08.383995 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f8018-37c3-49af-8c43-0a5de2f26dd1 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:08.489756 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f8018-37c3-49af-8c43-0a5de2f26dd1 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:08.504446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:08.526601 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e13fd-4b89-43fa-b55e-0684b32971bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:08.527247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e13fd-4b89-43fa-b55e-0684b32971bc None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:08.527525 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e13fd-4b89-43fa-b55e-0684b32971bc None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:08.527799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e13fd-4b89-43fa-b55e-0684b32971bc None admin] PATH_INFO: `/v3/users/4ea040edfa734126823bd4b21e953ac4` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:08.528927 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e13fd-4b89-43fa-b55e-0684b32971bc None admin] RBAC: Authorizing `identity:get_user(user_id=4ea040edfa734126823bd4b21e953ac4)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:08.530400 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e13fd-4b89-43fa-b55e-0684b32971bc None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:08.532156 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 153/311] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:09:08 2022] GET /identity/v3/users/4ea040edfa734126823bd4b21e953ac4 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:08.538459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744591c-6a39-4fbd-a89a-b623f085a6f0 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:08.591587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e13fd-4b89-43fa-b55e-0684b32971bc None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:08.653066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744591c-6a39-4fbd-a89a-b623f085a6f0 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:08.688256 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e30825-4961-4898-96a9-6ec163cbe224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:08.688922 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e30825-4961-4898-96a9-6ec163cbe224 None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:08.689990 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e30825-4961-4898-96a9-6ec163cbe224 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:08.690392 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e30825-4961-4898-96a9-6ec163cbe224 None admin] PATH_INFO: `/v3/system/users/4ea040edfa734126823bd4b21e953ac4/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:08.692066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e30825-4961-4898-96a9-6ec163cbe224 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4ea040edfa734126823bd4b21e953ac4, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:08.693464 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e30825-4961-4898-96a9-6ec163cbe224 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:08.703276 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e13fd-4b89-43fa-b55e-0684b32971bc None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:08.709567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 159/312] 10.222.0.60 () {58 vars in 1334 bytes} [Wed Sep 28 12:09:08 2022] PUT /identity/v3/system/users/4ea040edfa734126823bd4b21e953ac4/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:09:08.747071 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8759f1-11a6-4b8a-8012-547b0059a56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:08.747778 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8759f1-11a6-4b8a-8012-547b0059a56b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:08.748114 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8759f1-11a6-4b8a-8012-547b0059a56b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:08.748392 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8759f1-11a6-4b8a-8012-547b0059a56b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:08.761300 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8759f1-11a6-4b8a-8012-547b0059a56b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:08.767209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-6f8759f1-11a6-4b8a-8012-547b0059a56b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:09:08.767209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:08.767209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:08.767209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:08.767209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:08.767209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:08.767209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:08.767209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:08.767209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:08.767209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:08.767209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:08.767209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:08.767209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:08.767209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:08.767209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:08.767209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:08.768282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:08.768282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:08.768282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:08.768282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:08.768282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:08.768282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:08.768282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:08.768282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:08.768282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:08.768282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:08.768282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:08.768282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:08.768282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:08.768282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:08.768282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:08.768282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:08.769752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:08.769752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:08.769752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:08.769752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:08.769752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:08.769752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:08.769752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:08.769752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:08.769752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:08.769752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:08.769752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:09:08.769752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:08.769752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 154/313] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:08.774607 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e30825-4961-4898-96a9-6ec163cbe224 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:08.842491 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:08.896796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e30825-4961-4898-96a9-6ec163cbe224 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:08.933120 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443ac8b-a845-4086-9853-b59197a644b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:08.934125 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443ac8b-a845-4086-9853-b59197a644b7 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:08.934604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443ac8b-a845-4086-9853-b59197a644b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:08.935073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443ac8b-a845-4086-9853-b59197a644b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:08.935933 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443ac8b-a845-4086-9853-b59197a644b7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:08.936456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443ac8b-a845-4086-9853-b59197a644b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:08.938082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443ac8b-a845-4086-9853-b59197a644b7 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:08.946900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 160/314] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:09:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:08.952624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8759f1-11a6-4b8a-8012-547b0059a56b None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:09.009486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:09.069176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8759f1-11a6-4b8a-8012-547b0059a56b None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:09.104212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70362966-004b-434b-b7d3-3db90e124b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:09.104845 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70362966-004b-434b-b7d3-3db90e124b0e None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:09.105127 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70362966-004b-434b-b7d3-3db90e124b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:09.105426 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70362966-004b-434b-b7d3-3db90e124b0e None admin] PATH_INFO: `/v3/users/c0c2f447c91b455db5c8e72b7497f907` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:09.106555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70362966-004b-434b-b7d3-3db90e124b0e None admin] RBAC: Authorizing `identity:get_user(user_id=c0c2f447c91b455db5c8e72b7497f907)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:09.108057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70362966-004b-434b-b7d3-3db90e124b0e None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:09.109983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 155/315] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:09:08 2022] GET /identity/v3/users/c0c2f447c91b455db5c8e72b7497f907 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:09.115654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443ac8b-a845-4086-9853-b59197a644b7 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:09.223936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443ac8b-a845-4086-9853-b59197a644b7 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:09.241489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:09.259928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d99dce-f06a-4dba-bbea-3b39acf4dec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:09.260539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d99dce-f06a-4dba-bbea-3b39acf4dec2 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:09.260830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d99dce-f06a-4dba-bbea-3b39acf4dec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:09.261106 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d99dce-f06a-4dba-bbea-3b39acf4dec2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:09.329640 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d99dce-f06a-4dba-bbea-3b39acf4dec2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:09.331895 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d99dce-f06a-4dba-bbea-3b39acf4dec2 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:09.440003 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 161/316] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:09.445677 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70362966-004b-434b-b7d3-3db90e124b0e None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:09.552940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70362966-004b-434b-b7d3-3db90e124b0e None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:09.563432 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:09.587868 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82815719-83ff-4750-95c3-d755cf51fc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:09.588500 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82815719-83ff-4750-95c3-d755cf51fc76 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:09.588780 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82815719-83ff-4750-95c3-d755cf51fc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:09.589053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82815719-83ff-4750-95c3-d755cf51fc76 None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:09.590241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82815719-83ff-4750-95c3-d755cf51fc76 None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:09.591672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82815719-83ff-4750-95c3-d755cf51fc76 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:09.597240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 156/317] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:09 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:09.600325 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d99dce-f06a-4dba-bbea-3b39acf4dec2 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:09.618821 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:09.705794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d99dce-f06a-4dba-bbea-3b39acf4dec2 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:09.740838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38ce43-cef0-4359-a18b-19561eec3e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:09.741475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38ce43-cef0-4359-a18b-19561eec3e62 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:09.741755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38ce43-cef0-4359-a18b-19561eec3e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:09.742028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38ce43-cef0-4359-a18b-19561eec3e62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:09.742751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38ce43-cef0-4359-a18b-19561eec3e62 None admin] RBAC: Adding query filter params (role.id=edf27bedabd24681b24ec2a72a774a9c, scope.project.id=81a7733885564252af45b01ea11bfed2, user.id=c0c2f447c91b455db5c8e72b7497f907) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:09.743057 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38ce43-cef0-4359-a18b-19561eec3e62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:09.744477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38ce43-cef0-4359-a18b-19561eec3e62 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:09.755808 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 162/318] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:09:09 2022] GET /identity/v3/role_assignments?user.id=c0c2f447c91b455db5c8e72b7497f907&scope.project.id=81a7733885564252af45b01ea11bfed2&role.id=edf27bedabd24681b24ec2a72a774a9c => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:10.335391 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0eb26-ff26-46ae-ad64-8b530d10dcc7 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:10.335787 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0eb26-ff26-46ae-ad64-8b530d10dcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:10.335979 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a0eb26-ff26-46ae-ad64-8b530d10dcc7 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:10.337456 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 157/319] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:10.342513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727ea14-2751-44bd-a363-9ecd3d2be925 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:10.342805 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727ea14-2751-44bd-a363-9ecd3d2be925 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:10.343083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727ea14-2751-44bd-a363-9ecd3d2be925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:10.362108 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-8727ea14-2751-44bd-a363-9ecd3d2be925 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:10.363335 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8727ea14-2751-44bd-a363-9ecd3d2be925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:10.436653 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 163/320] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:10.491142 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a6ae4-9f5e-4f33-8f72-6f731c94085c None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:10.491655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a6ae4-9f5e-4f33-8f72-6f731c94085c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:10.491727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a6ae4-9f5e-4f33-8f72-6f731c94085c None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:10.493154 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 158/321] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:10.498411 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2ed2e-aef7-4b14-b682-d0b440c78b1a None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:10.498698 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2ed2e-aef7-4b14-b682-d0b440c78b1a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:10.498969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2ed2e-aef7-4b14-b682-d0b440c78b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:10.517214 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-63d2ed2e-aef7-4b14-b682-d0b440c78b1a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:10.518553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-63d2ed2e-aef7-4b14-b682-d0b440c78b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:10.526246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:10.587594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 164/322] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:10.707703 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a6ae4-9f5e-4f33-8f72-6f731c94085c None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:10.811678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a6ae4-9f5e-4f33-8f72-6f731c94085c None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:10.849300 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b7de9-08c4-42aa-8302-9670a9e3cd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:10.849957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b7de9-08c4-42aa-8302-9670a9e3cd2c None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:10.850237 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b7de9-08c4-42aa-8302-9670a9e3cd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:10.850509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b7de9-08c4-42aa-8302-9670a9e3cd2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:10.851647 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b7de9-08c4-42aa-8302-9670a9e3cd2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:10.853094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b7de9-08c4-42aa-8302-9670a9e3cd2c None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:10.854894 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 159/323] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:10.859146 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2ed2e-aef7-4b14-b682-d0b440c78b1a None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:10.860709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b7de9-08c4-42aa-8302-9670a9e3cd2c None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:10.912129 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:10.962255 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2ed2e-aef7-4b14-b682-d0b440c78b1a None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:10.968239 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b7de9-08c4-42aa-8302-9670a9e3cd2c None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:10.996004 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17124567-8c51-4cf3-9cda-67d055d45e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:10.996634 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17124567-8c51-4cf3-9cda-67d055d45e95 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:10.996914 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17124567-8c51-4cf3-9cda-67d055d45e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:10.997286 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17124567-8c51-4cf3-9cda-67d055d45e95 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:11.001158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1627fea-0bf4-488c-9f89-9fb26d97832f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:11.002306 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1627fea-0bf4-488c-9f89-9fb26d97832f None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:11.002794 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1627fea-0bf4-488c-9f89-9fb26d97832f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:11.003067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1627fea-0bf4-488c-9f89-9fb26d97832f None admin] PATH_INFO: `/v3/groups` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:11.003678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1627fea-0bf4-488c-9f89-9fb26d97832f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:11.005097 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1627fea-0bf4-488c-9f89-9fb26d97832f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:11.007759 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17124567-8c51-4cf3-9cda-67d055d45e95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:11.012628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-17124567-8c51-4cf3-9cda-67d055d45e95 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:09:11.012628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:11.012628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:11.012628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:11.012628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:11.012628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:11.012628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:11.012628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:11.012628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:11.012628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:11.012628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:11.012628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:11.012628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:11.012628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:11.012628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:11.012628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:11.013765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:11.013765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:11.013765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:11.013765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:11.013765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:11.013765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:11.013765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:11.013765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:11.013765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:11.013765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:11.013765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:11.013765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:11.013765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:11.013765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:11.013765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:11.013765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:11.014558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:11.014558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:11.014558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:11.014558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:11.014558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:11.014558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:11.014558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:11.014558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:11.014558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:11.014558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:11.014558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:09:11.014558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:11.014558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 165/324] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:11.015222 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eee4cb-be9f-47df-93f1-3ca7a463976d None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:11.015285 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eee4cb-be9f-47df-93f1-3ca7a463976d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:11.015671 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eee4cb-be9f-47df-93f1-3ca7a463976d None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:11.016915 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 166/325] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:11.020090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eee4cb-be9f-47df-93f1-3ca7a463976d None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:11.024612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:11.024953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 160/326] 10.222.0.60 () {60 vars in 1134 bytes} [Wed Sep 28 12:09:10 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:09:11.026293 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad777c-2f6f-413c-bfd2-755c9b69583a None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:11.026948 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad777c-2f6f-413c-bfd2-755c9b69583a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:11.026948 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad777c-2f6f-413c-bfd2-755c9b69583a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:11.047158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-e4ad777c-2f6f-413c-bfd2-755c9b69583a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:11.048372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e4ad777c-2f6f-413c-bfd2-755c9b69583a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:11.129687 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 161/327] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:11.143500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eee4cb-be9f-47df-93f1-3ca7a463976d None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:11.193027 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbdcd49-5037-4691-997b-2de666e61c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:11.193688 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbdcd49-5037-4691-997b-2de666e61c07 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:11.193966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbdcd49-5037-4691-997b-2de666e61c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:11.194236 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbdcd49-5037-4691-997b-2de666e61c07 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:11.194909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbdcd49-5037-4691-997b-2de666e61c07 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:11.195203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbdcd49-5037-4691-997b-2de666e61c07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:11.196578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbdcd49-5037-4691-997b-2de666e61c07 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:11.206809 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 167/328] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:09:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:11.212096 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad777c-2f6f-413c-bfd2-755c9b69583a None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:11.255013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:11.329809 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad777c-2f6f-413c-bfd2-755c9b69583a None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:11.367446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c43f2-7170-49d0-8342-779fb71db2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:11.368082 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c43f2-7170-49d0-8342-779fb71db2ae None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:11.368359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c43f2-7170-49d0-8342-779fb71db2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:11.368625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c43f2-7170-49d0-8342-779fb71db2ae None admin] PATH_INFO: `/v3/users/31ff84c2f79241c09d73f2abddaadbcc` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:11.369739 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c43f2-7170-49d0-8342-779fb71db2ae None admin] RBAC: Authorizing `identity:get_user(user_id=31ff84c2f79241c09d73f2abddaadbcc)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:11.371166 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c43f2-7170-49d0-8342-779fb71db2ae None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:11.372887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 162/329] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:09:11 2022] GET /identity/v3/users/31ff84c2f79241c09d73f2abddaadbcc => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:11.378804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbdcd49-5037-4691-997b-2de666e61c07 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:11.400762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c43f2-7170-49d0-8342-779fb71db2ae None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:11.457878 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:11.508801 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbdcd49-5037-4691-997b-2de666e61c07 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:11.523764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c43f2-7170-49d0-8342-779fb71db2ae None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:11.526186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:11.550238 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c783c-b951-41eb-88f8-f51b1737cf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:11.550897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c783c-b951-41eb-88f8-f51b1737cf36 None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:11.551172 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c783c-b951-41eb-88f8-f51b1737cf36 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:11.551443 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c783c-b951-41eb-88f8-f51b1737cf36 None admin] PATH_INFO: `/v3/system/users/31ff84c2f79241c09d73f2abddaadbcc/roles/edf27bedabd24681b24ec2a72a774a9c` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:11.554248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86c783c-b951-41eb-88f8-f51b1737cf36 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=31ff84c2f79241c09d73f2abddaadbcc, role_id=edf27bedabd24681b24ec2a72a774a9c)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:11.556535 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86c783c-b951-41eb-88f8-f51b1737cf36 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:11.567381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 168/330] 10.222.0.60 () {58 vars in 1334 bytes} [Wed Sep 28 12:09:11 2022] PUT /identity/v3/system/users/31ff84c2f79241c09d73f2abddaadbcc/roles/edf27bedabd24681b24ec2a72a774a9c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:09:11.573264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108233d-9f43-44df-ab16-1042072fb545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:11.573264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108233d-9f43-44df-ab16-1042072fb545 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:11.573264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108233d-9f43-44df-ab16-1042072fb545 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:11.573264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108233d-9f43-44df-ab16-1042072fb545 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:11.592206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108233d-9f43-44df-ab16-1042072fb545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:11.598673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-e108233d-9f43-44df-ab16-1042072fb545 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:11.598673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:11.598673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:11.598673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:11.598673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:11.598673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:11.598673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:11.598673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:11.598673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:11.598673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:11.598673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:11.598673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:11.598673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:11.598673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:11.598673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:11.598673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:11.599810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:11.599810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:11.599810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:11.599810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:11.599810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:11.599810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:11.599810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:11.599810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:11.599810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:11.599810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:11.599810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:11.599810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:11.599810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:11.599810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:11.599810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:11.599810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:11.600822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:11.600822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:11.600822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:11.600822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:11.600822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:11.600822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:11.600822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:11.600822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:11.600822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:11.600822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:11.600822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:11.600822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:11.603251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 163/331] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:11.609449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c783c-b951-41eb-88f8-f51b1737cf36 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:11.738036 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c783c-b951-41eb-88f8-f51b1737cf36 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:11.772513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:11.781574 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facfe6c-40ab-43e7-8c48-8d80da9ec73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:11.782622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facfe6c-40ab-43e7-8c48-8d80da9ec73a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:11.783898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facfe6c-40ab-43e7-8c48-8d80da9ec73a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:11.784293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0facfe6c-40ab-43e7-8c48-8d80da9ec73a None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:11.784982 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0facfe6c-40ab-43e7-8c48-8d80da9ec73a None admin] RBAC: Adding query filter params (name=member) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:11.785680 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0facfe6c-40ab-43e7-8c48-8d80da9ec73a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:11.787156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0facfe6c-40ab-43e7-8c48-8d80da9ec73a None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:11.796589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 169/332] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:09:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:11.802232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108233d-9f43-44df-ab16-1042072fb545 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:11.926751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108233d-9f43-44df-ab16-1042072fb545 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:11.964328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab939a-3086-4c16-9044-a7995c6435f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:11.964964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab939a-3086-4c16-9044-a7995c6435f0 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:11.965311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab939a-3086-4c16-9044-a7995c6435f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:11.965591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab939a-3086-4c16-9044-a7995c6435f0 None admin] PATH_INFO: `/v3/users/4ea040edfa734126823bd4b21e953ac4` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:11.966792 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab939a-3086-4c16-9044-a7995c6435f0 None admin] RBAC: Authorizing `identity:get_user(user_id=4ea040edfa734126823bd4b21e953ac4)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:11.968287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab939a-3086-4c16-9044-a7995c6435f0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:11.970023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 164/333] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:09:11 2022] GET /identity/v3/users/4ea040edfa734126823bd4b21e953ac4 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:11.976585 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facfe6c-40ab-43e7-8c48-8d80da9ec73a None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:12.098574 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0facfe6c-40ab-43e7-8c48-8d80da9ec73a None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:12.143559 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf2478-ae69-461e-af59-80c02a24b272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:12.144178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf2478-ae69-461e-af59-80c02a24b272 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:12.144501 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf2478-ae69-461e-af59-80c02a24b272 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:12.144897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf2478-ae69-461e-af59-80c02a24b272 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:12.145666 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf2478-ae69-461e-af59-80c02a24b272 None admin] RBAC: Adding query filter params (role.id=2201e005d5ef449896a0e68a940678ec, user.id=4ea040edfa734126823bd4b21e953ac4, scope.system=all) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:12.146024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf2478-ae69-461e-af59-80c02a24b272 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:12.147479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf2478-ae69-461e-af59-80c02a24b272 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:12.172320 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 170/334] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Sep 28 12:09:11 2022] GET /identity/v3/role_assignments?user.id=4ea040edfa734126823bd4b21e953ac4&scope.system=all&role.id=2201e005d5ef449896a0e68a940678ec => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:13.618559 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0a85d-33e1-486f-b10d-bf10874074fa None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:13.619104 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0a85d-33e1-486f-b10d-bf10874074fa None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:13.619104 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0a85d-33e1-486f-b10d-bf10874074fa None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:13.620684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 165/335] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:13.626296 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd98e4-eea6-4338-9cae-7bcaf1a3d480 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:13.627239 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd98e4-eea6-4338-9cae-7bcaf1a3d480 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:13.627434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd98e4-eea6-4338-9cae-7bcaf1a3d480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:13.647569 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e6dd98e4-eea6-4338-9cae-7bcaf1a3d480 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:13.648769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e6dd98e4-eea6-4338-9cae-7bcaf1a3d480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:13.703290 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:13.735454 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 171/336] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:14.290897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0a85d-33e1-486f-b10d-bf10874074fa None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:14.324702 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:14.407791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0a85d-33e1-486f-b10d-bf10874074fa None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:14.450257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b24be-5c6d-4881-8eed-735bff840ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:14.450909 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b24be-5c6d-4881-8eed-735bff840ae3 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:14.451189 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b24be-5c6d-4881-8eed-735bff840ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:14.451461 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b24be-5c6d-4881-8eed-735bff840ae3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:14.463550 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b24be-5c6d-4881-8eed-735bff840ae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:14.468731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-615b24be-5c6d-4881-8eed-735bff840ae3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:14.468731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:14.468731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:14.468731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:14.468731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:14.468731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:14.468731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:14.468731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:14.468731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:14.468731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:14.468731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:14.468731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:14.468731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:14.468731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:14.468731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:14.468731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:14.469808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:14.469808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:14.469808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:14.469808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:14.469808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:14.469808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:14.469808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:14.469808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:14.469808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:14.469808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:14.469808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:14.469808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:14.469808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:14.469808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:14.469808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:14.469808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:14.471458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:14.471458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:14.471458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:14.471458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:14.471458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:14.471458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:14.471458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:14.471458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:14.471458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:14.471458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:14.471458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:14.471458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:14.471458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 166/337] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:14.476936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd98e4-eea6-4338-9cae-7bcaf1a3d480 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:14.574549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7eb67-d3b5-4aa2-a1c1-9187f842f08c None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:14.574865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7eb67-d3b5-4aa2-a1c1-9187f842f08c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:14.575095 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7eb67-d3b5-4aa2-a1c1-9187f842f08c None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:14.576534 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 167/338] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:14.590837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169155d6-ca36-4e27-a5d0-7ad10190ba55 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:14.591432 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169155d6-ca36-4e27-a5d0-7ad10190ba55 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:14.591982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169155d6-ca36-4e27-a5d0-7ad10190ba55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:14.592414 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd98e4-eea6-4338-9cae-7bcaf1a3d480 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:14.622199 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-169155d6-ca36-4e27-a5d0-7ad10190ba55 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:14.624131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-169155d6-ca36-4e27-a5d0-7ad10190ba55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:14.632834 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b2881-21ff-4777-92f8-0933b81a05e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:14.633479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b2881-21ff-4777-92f8-0933b81a05e4 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:14.633757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b2881-21ff-4777-92f8-0933b81a05e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:14.634024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b2881-21ff-4777-92f8-0933b81a05e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:14.634690 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b2881-21ff-4777-92f8-0933b81a05e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:14.634987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b2881-21ff-4777-92f8-0933b81a05e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:14.636354 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b2881-21ff-4777-92f8-0933b81a05e4 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:14.647720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 172/339] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:09:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:14.655075 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b2881-21ff-4777-92f8-0933b81a05e4 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:14.739949 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 168/340] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:14.741891 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ec517-4cd9-4cd6-8370-b08c244b561d None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:14.741891 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ec517-4cd9-4cd6-8370-b08c244b561d None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:14.741891 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ec517-4cd9-4cd6-8370-b08c244b561d None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:14.743369 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 169/341] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:14.749464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2d8bd-9e95-4fce-ab5f-47bcb4f2a915 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:14.749744 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2d8bd-9e95-4fce-ab5f-47bcb4f2a915 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:14.750073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2d8bd-9e95-4fce-ab5f-47bcb4f2a915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:14.752257 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:14.773036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-36c2d8bd-9e95-4fce-ab5f-47bcb4f2a915 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:14.774241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-36c2d8bd-9e95-4fce-ab5f-47bcb4f2a915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:14.793694 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:14.860697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 170/342] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:09:14.862980 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e404df-6770-47fe-bab7-82145ceeae01 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:14.863264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e404df-6770-47fe-bab7-82145ceeae01 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:14.863535 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e404df-6770-47fe-bab7-82145ceeae01 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:14.864894 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 171/343] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:14.870594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28742085-e56f-4cde-b706-ab4ebbfab76c None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:14.870874 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28742085-e56f-4cde-b706-ab4ebbfab76c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:14.871141 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28742085-e56f-4cde-b706-ab4ebbfab76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:14.895349 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-28742085-e56f-4cde-b706-ab4ebbfab76c None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:14.896518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-28742085-e56f-4cde-b706-ab4ebbfab76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:14.943379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b2881-21ff-4777-92f8-0933b81a05e4 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:14.984326 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 172/344] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:15.016954 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f09090-16e8-4178-b993-a999477d791f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:15.017716 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f09090-16e8-4178-b993-a999477d791f None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:15.018196 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f09090-16e8-4178-b993-a999477d791f None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:15.018481 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f09090-16e8-4178-b993-a999477d791f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:15.124611 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f09090-16e8-4178-b993-a999477d791f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:15.126106 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f09090-16e8-4178-b993-a999477d791f None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:15.197520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:15.246132 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 173/345] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:15.247964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28742085-e56f-4cde-b706-ab4ebbfab76c None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:15.251520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f09090-16e8-4178-b993-a999477d791f None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:15.354980 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:15.363610 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28742085-e56f-4cde-b706-ab4ebbfab76c None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:15.401519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7712b4a-2891-42cb-819a-5ceeea7f4672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:15.402160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7712b4a-2891-42cb-819a-5ceeea7f4672 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:15.402747 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7712b4a-2891-42cb-819a-5ceeea7f4672 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:15.403034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7712b4a-2891-42cb-819a-5ceeea7f4672 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:15.414582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7712b4a-2891-42cb-819a-5ceeea7f4672 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:15.420382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-e7712b4a-2891-42cb-819a-5ceeea7f4672 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:09:15.420382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:15.420382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:15.420382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:15.420382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:15.420382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:15.420382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:15.420382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:15.420382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:15.420382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:15.420382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:15.420382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:15.420382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:15.420382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:15.420382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:15.420382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:15.421303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:15.421303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:15.421303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:15.421303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:15.421303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:15.421303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:15.421303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:15.421303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:15.421303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:15.421303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:15.421303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:15.421303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:15.421303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:15.421303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:15.421303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:15.421303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:15.422327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:15.422327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:15.422327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:15.422327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:15.422327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:15.422327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:15.422327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:15.422327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:15.422327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:15.422327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:15.422327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 12:09:15.422327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:15.422327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 173/346] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:15.422963 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7712b4a-2891-42cb-819a-5ceeea7f4672 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:15.477243 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f09090-16e8-4178-b993-a999477d791f None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:15.515937 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbc8bc-0fcb-442d-9765-1bd0f9e48fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:15.516635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbc8bc-0fcb-442d-9765-1bd0f9e48fce None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:15.516923 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbc8bc-0fcb-442d-9765-1bd0f9e48fce None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:15.518133 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbc8bc-0fcb-442d-9765-1bd0f9e48fce None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:15.521016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbc8bc-0fcb-442d-9765-1bd0f9e48fce None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:15.522559 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbc8bc-0fcb-442d-9765-1bd0f9e48fce None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:15.524528 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 174/347] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:15 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:15.529445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbc8bc-0fcb-442d-9765-1bd0f9e48fce None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:15.540832 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7712b4a-2891-42cb-819a-5ceeea7f4672 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:15.558270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:15.589053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25bf6ad-6881-4041-bd6e-9faeaa6c5eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:15.589730 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bf6ad-6881-4041-bd6e-9faeaa6c5eec None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:15.590014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bf6ad-6881-4041-bd6e-9faeaa6c5eec None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:15.590291 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25bf6ad-6881-4041-bd6e-9faeaa6c5eec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:15.602896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25bf6ad-6881-4041-bd6e-9faeaa6c5eec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:15.607916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-b25bf6ad-6881-4041-bd6e-9faeaa6c5eec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:09:15.607916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:15.607916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:15.607916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:15.607916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:15.607916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:15.607916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:15.607916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:15.607916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:15.607916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:15.607916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:15.607916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:15.607916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:15.607916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:15.607916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:15.607916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:15.608800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:15.608800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:15.608800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:15.608800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:15.608800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:15.608800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:15.608800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:15.608800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:15.608800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:15.608800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:15.608800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:15.608800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:15.608800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:15.608800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:15.608800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:15.608800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:15.610022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:15.610022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:15.610022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:15.610022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:15.610022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:15.610022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:15.610022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:15.610022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:15.610022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:15.610022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:15.610022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:09:15.610022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:15.610022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 174/348] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:09:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:15.610022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25bf6ad-6881-4041-bd6e-9faeaa6c5eec None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:15.616812 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:15.663768 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbc8bc-0fcb-442d-9765-1bd0f9e48fce None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:15.707108 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a6fc1-ec0b-4ac3-b9db-d6f744389749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:15.708113 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a6fc1-ec0b-4ac3-b9db-d6f744389749 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:15.708397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a6fc1-ec0b-4ac3-b9db-d6f744389749 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:15.708672 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a6fc1-ec0b-4ac3-b9db-d6f744389749 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:15.709637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a6fc1-ec0b-4ac3-b9db-d6f744389749 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:15.710625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a6fc1-ec0b-4ac3-b9db-d6f744389749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:15.712019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a6fc1-ec0b-4ac3-b9db-d6f744389749 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:15.722706 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 175/349] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:09:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:15.723565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a6fc1-ec0b-4ac3-b9db-d6f744389749 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:15.743933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25bf6ad-6881-4041-bd6e-9faeaa6c5eec None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:15.790596 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f730f-664a-4747-b000-423a5ee04725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:15.791501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f730f-664a-4747-b000-423a5ee04725 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:15.791922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f730f-664a-4747-b000-423a5ee04725 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:15.792362 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6f730f-664a-4747-b000-423a5ee04725 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:15.809251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6f730f-664a-4747-b000-423a5ee04725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:15.817329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:15.818302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-4b6f730f-664a-4747-b000-423a5ee04725 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:15.818302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:15.818302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:15.818302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:15.818302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:15.818302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:15.818302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:15.818302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:15.818302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:15.818302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:15.818302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:15.818302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:15.818302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:15.818302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:15.818302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:15.818302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:15.819348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:15.819348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:15.819348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:15.819348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:15.819348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:15.819348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:15.819348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:15.819348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:15.819348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:15.819348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:15.819348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:15.819348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:15.819348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:15.819348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:15.819348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:15.819348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:15.820131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:15.820131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:15.820131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:15.820131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:15.820131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:15.820131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:15.820131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:15.820131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:15.820131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:15.820131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:15.820131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:15.820131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:15.823936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 175/350] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:15.825013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f730f-664a-4747-b000-423a5ee04725 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:15.862574 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a6fc1-ec0b-4ac3-b9db-d6f744389749 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:15.908839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:15.912468 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8a431-33b1-4ebc-a8e5-aa05de88750f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:15.913375 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8a431-33b1-4ebc-a8e5-aa05de88750f None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:15.914037 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8a431-33b1-4ebc-a8e5-aa05de88750f None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:15.914435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8a431-33b1-4ebc-a8e5-aa05de88750f None admin] PATH_INFO: `/v3/groups/9394c6da936942d7814fb34ce09df006` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:15.922092 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8a431-33b1-4ebc-a8e5-aa05de88750f None admin] RBAC: Authorizing `identity:get_group(group_id=9394c6da936942d7814fb34ce09df006)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:15.924513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8a431-33b1-4ebc-a8e5-aa05de88750f None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:15.926764 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 176/351] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 12:09:15 2022] GET /identity/v3/groups/9394c6da936942d7814fb34ce09df006 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:15.927829 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8a431-33b1-4ebc-a8e5-aa05de88750f None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:15.954049 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6f730f-664a-4747-b000-423a5ee04725 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:15.995169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db3676-e942-4d1b-a467-e1e46b0db50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:15.995893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db3676-e942-4d1b-a467-e1e46b0db50e None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:15.996599 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db3676-e942-4d1b-a467-e1e46b0db50e None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:15.996957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db3676-e942-4d1b-a467-e1e46b0db50e None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:15.997918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db3676-e942-4d1b-a467-e1e46b0db50e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:15.998227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db3676-e942-4d1b-a467-e1e46b0db50e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:16.000253 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db3676-e942-4d1b-a467-e1e46b0db50e None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:16.010059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 176/352] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:16.011323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db3676-e942-4d1b-a467-e1e46b0db50e None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:16.050761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8a431-33b1-4ebc-a8e5-aa05de88750f None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:16.099570 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0763f4-3d59-4b7a-a75c-49d3b9851a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:16.100596 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0763f4-3d59-4b7a-a75c-49d3b9851a8f None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:16.100953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0763f4-3d59-4b7a-a75c-49d3b9851a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:16.101178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0763f4-3d59-4b7a-a75c-49d3b9851a8f None admin] PATH_INFO: `/v3/users/31ff84c2f79241c09d73f2abddaadbcc` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:16.102417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0763f4-3d59-4b7a-a75c-49d3b9851a8f None admin] RBAC: Authorizing `identity:get_user(user_id=31ff84c2f79241c09d73f2abddaadbcc)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:16.103908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0763f4-3d59-4b7a-a75c-49d3b9851a8f None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:16.106594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 177/353] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:09:15 2022] GET /identity/v3/users/31ff84c2f79241c09d73f2abddaadbcc => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:16.107785 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e0273-e31d-4a17-a769-9f251f85b9b4 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:16.108071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e0273-e31d-4a17-a769-9f251f85b9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:16.108353 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e0273-e31d-4a17-a769-9f251f85b9b4 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:16.109801 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 178/354] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:16.110371 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e0273-e31d-4a17-a769-9f251f85b9b4 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:16.113559 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:16.141615 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db3676-e942-4d1b-a467-e1e46b0db50e None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:16.181920 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:16.194672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5384b8-b37e-46be-bf2b-681fe77bc7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:16.195295 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5384b8-b37e-46be-bf2b-681fe77bc7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:16.195571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5384b8-b37e-46be-bf2b-681fe77bc7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:16.195846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5384b8-b37e-46be-bf2b-681fe77bc7a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:16.196509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5384b8-b37e-46be-bf2b-681fe77bc7a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:16.196811 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5384b8-b37e-46be-bf2b-681fe77bc7a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:16.198326 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5384b8-b37e-46be-bf2b-681fe77bc7a2 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:16.207438 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 177/355] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:09:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:16.208353 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5384b8-b37e-46be-bf2b-681fe77bc7a2 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:16.254165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e0273-e31d-4a17-a769-9f251f85b9b4 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:16.297527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07791ea5-386d-4de8-9608-5679c6675d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:16.298177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07791ea5-386d-4de8-9608-5679c6675d17 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:16.298982 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07791ea5-386d-4de8-9608-5679c6675d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:16.299261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07791ea5-386d-4de8-9608-5679c6675d17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:16.299992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07791ea5-386d-4de8-9608-5679c6675d17 None admin] RBAC: Adding query filter params (group.id=9394c6da936942d7814fb34ce09df006, role.id=2201e005d5ef449896a0e68a940678ec, scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:16.300302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07791ea5-386d-4de8-9608-5679c6675d17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:16.301790 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07791ea5-386d-4de8-9608-5679c6675d17 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:16.308075 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 179/356] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 12:09:16 2022] GET /identity/v3/role_assignments?group.id=9394c6da936942d7814fb34ce09df006&scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3&role.id=2201e005d5ef449896a0e68a940678ec => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:16.308676 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00a6ce-c63c-4a8e-a341-101644c0df8d None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:16.309088 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00a6ce-c63c-4a8e-a341-101644c0df8d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:16.309649 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00a6ce-c63c-4a8e-a341-101644c0df8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:16.330547 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-8f00a6ce-c63c-4a8e-a341-101644c0df8d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:16.332103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8f00a6ce-c63c-4a8e-a341-101644c0df8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:16.344047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5384b8-b37e-46be-bf2b-681fe77bc7a2 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:16.399526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a7eb9-ad8b-4738-b9a7-65f0c77c7809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:16.400070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a7eb9-ad8b-4738-b9a7-65f0c77c7809 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:16.400531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a7eb9-ad8b-4738-b9a7-65f0c77c7809 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:16.400813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a7eb9-ad8b-4738-b9a7-65f0c77c7809 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:16.418248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:16.427421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 180/357] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:16.429331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00a6ce-c63c-4a8e-a341-101644c0df8d None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:16.487107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:16.509434 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a7eb9-ad8b-4738-b9a7-65f0c77c7809 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:16.510928 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a7eb9-ad8b-4738-b9a7-65f0c77c7809 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:16.551585 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00a6ce-c63c-4a8e-a341-101644c0df8d None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:16.589699 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb648f70-fad6-4f7f-b37b-9717f1ae5daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:16.590469 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb648f70-fad6-4f7f-b37b-9717f1ae5daa None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:16.590747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb648f70-fad6-4f7f-b37b-9717f1ae5daa None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:16.591329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb648f70-fad6-4f7f-b37b-9717f1ae5daa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:16.592334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb648f70-fad6-4f7f-b37b-9717f1ae5daa None admin] RBAC: Adding query filter params (role.id=edf27bedabd24681b24ec2a72a774a9c, user.id=31ff84c2f79241c09d73f2abddaadbcc, scope.system=all) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:16.592647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb648f70-fad6-4f7f-b37b-9717f1ae5daa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:16.594392 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb648f70-fad6-4f7f-b37b-9717f1ae5daa None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:16.614535 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 181/358] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Sep 28 12:09:16 2022] GET /identity/v3/role_assignments?user.id=31ff84c2f79241c09d73f2abddaadbcc&scope.system=all&role.id=edf27bedabd24681b24ec2a72a774a9c => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:16.615188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb648f70-fad6-4f7f-b37b-9717f1ae5daa None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:16.645466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 178/359] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:16.646785 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e04ead-051b-46af-bc10-76566e7c4c43 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:16.647070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e04ead-051b-46af-bc10-76566e7c4c43 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:16.647376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e04ead-051b-46af-bc10-76566e7c4c43 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:16.653667 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 179/360] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:16.654640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e04ead-051b-46af-bc10-76566e7c4c43 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:16.679002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:16.751692 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb648f70-fad6-4f7f-b37b-9717f1ae5daa None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:16.778604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:16.799555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e04ead-051b-46af-bc10-76566e7c4c43 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:16.816027 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805784a-a9e6-4fb5-81f1-810a394982f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:16.816710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805784a-a9e6-4fb5-81f1-810a394982f7 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:16.816999 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805784a-a9e6-4fb5-81f1-810a394982f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:16.817300 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805784a-a9e6-4fb5-81f1-810a394982f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:16.844315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2441e-d7e8-4afa-af11-2563bdcd04e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:16.844969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2441e-d7e8-4afa-af11-2563bdcd04e8 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:16.845511 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2441e-d7e8-4afa-af11-2563bdcd04e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:16.845785 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2441e-d7e8-4afa-af11-2563bdcd04e8 None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:16.847702 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2441e-d7e8-4afa-af11-2563bdcd04e8 None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:16.849174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2441e-d7e8-4afa-af11-2563bdcd04e8 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:16.851079 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 180/361] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:16 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:16.852102 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c1143-e944-434a-9a48-d111bdf76a95 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:16.852465 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c1143-e944-434a-9a48-d111bdf76a95 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:16.852639 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c1143-e944-434a-9a48-d111bdf76a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:16.885757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-ea6c1143-e944-434a-9a48-d111bdf76a95 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:16.886970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-ea6c1143-e944-434a-9a48-d111bdf76a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:16.933727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0805784a-a9e6-4fb5-81f1-810a394982f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:16.935217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0805784a-a9e6-4fb5-81f1-810a394982f7 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:16.966917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 181/362] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:16.967916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c1143-e944-434a-9a48-d111bdf76a95 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:17.006376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:17.072528 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 182/363] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:17.073644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805784a-a9e6-4fb5-81f1-810a394982f7 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:17.076983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:17.097712 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c1143-e944-434a-9a48-d111bdf76a95 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:17.136944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8601f5-c49c-4a99-9b95-6c85d67eb40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:17.137819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8601f5-c49c-4a99-9b95-6c85d67eb40d None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:17.138106 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8601f5-c49c-4a99-9b95-6c85d67eb40d None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:17.138377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8601f5-c49c-4a99-9b95-6c85d67eb40d None admin] PATH_INFO: `/v3/groups/9394c6da936942d7814fb34ce09df006` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:17.140176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8601f5-c49c-4a99-9b95-6c85d67eb40d None admin] RBAC: Authorizing `identity:get_group(group_id=9394c6da936942d7814fb34ce09df006)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:17.141733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8601f5-c49c-4a99-9b95-6c85d67eb40d None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:17.144512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 182/364] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 12:09:16 2022] GET /identity/v3/groups/9394c6da936942d7814fb34ce09df006 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:17.145372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8601f5-c49c-4a99-9b95-6c85d67eb40d None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:17.226349 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805784a-a9e6-4fb5-81f1-810a394982f7 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:17.270548 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5775b-97fb-4ffa-8269-54a24bfed48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:17.271564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5775b-97fb-4ffa-8269-54a24bfed48e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:17.272034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5775b-97fb-4ffa-8269-54a24bfed48e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:17.272481 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5775b-97fb-4ffa-8269-54a24bfed48e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:17.284749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8601f5-c49c-4a99-9b95-6c85d67eb40d None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:17.285252 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5775b-97fb-4ffa-8269-54a24bfed48e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:17.290517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-82c5775b-97fb-4ffa-8269-54a24bfed48e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:09:17.290517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:17.290517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:17.290517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:17.290517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:17.290517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:17.290517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:17.290517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:17.290517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:17.290517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:17.290517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:17.290517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:17.290517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:17.290517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:17.290517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:17.290517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:17.291399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:17.291399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:17.291399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:17.291399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:17.291399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:17.291399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:17.291399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:17.291399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:17.291399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:17.291399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:17.291399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:17.291399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:17.291399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:17.291399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:17.291399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:17.291399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:17.292207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:17.292207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:17.292207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:17.292207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:17.292207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:17.292207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:17.292207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:17.292207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:17.292207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:17.292207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:17.292207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:09:17.292207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:17.294875 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 183/365] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:09:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:17.295834 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5775b-97fb-4ffa-8269-54a24bfed48e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:17.318959 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:17.333930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912de90-3836-4b6e-81a9-772088f889ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:17.334586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912de90-3836-4b6e-81a9-772088f889ae None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:17.334861 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912de90-3836-4b6e-81a9-772088f889ae None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:17.335129 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912de90-3836-4b6e-81a9-772088f889ae None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:17.337886 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912de90-3836-4b6e-81a9-772088f889ae None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:17.339398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912de90-3836-4b6e-81a9-772088f889ae None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:17.341095 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 183/366] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:17 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:17.342989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912de90-3836-4b6e-81a9-772088f889ae None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:17.371530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:17.435098 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5775b-97fb-4ffa-8269-54a24bfed48e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:17.478999 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912de90-3836-4b6e-81a9-772088f889ae None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:17.482888 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfccbbf-3f20-48ba-af94-3715daab93d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:17.483703 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfccbbf-3f20-48ba-af94-3715daab93d4 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:17.484161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfccbbf-3f20-48ba-af94-3715daab93d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:17.484605 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfccbbf-3f20-48ba-af94-3715daab93d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:17.485566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfccbbf-3f20-48ba-af94-3715daab93d4 None admin] RBAC: Adding query filter params (group.id=9394c6da936942d7814fb34ce09df006, role.id=8e9576d16f194938b86a1d1e087895bd, scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:17.486947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfccbbf-3f20-48ba-af94-3715daab93d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:17.488574 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfccbbf-3f20-48ba-af94-3715daab93d4 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:17.498054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 184/367] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 12:09:17 2022] GET /identity/v3/role_assignments?group.id=9394c6da936942d7814fb34ce09df006&scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3&role.id=8e9576d16f194938b86a1d1e087895bd => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:17.499872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfccbbf-3f20-48ba-af94-3715daab93d4 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:17.523749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc875bb-168d-4310-8bdb-0256180d9079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:17.524697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc875bb-168d-4310-8bdb-0256180d9079 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:17.525178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc875bb-168d-4310-8bdb-0256180d9079 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:17.525655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc875bb-168d-4310-8bdb-0256180d9079 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:17.526523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc875bb-168d-4310-8bdb-0256180d9079 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:17.527001 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc875bb-168d-4310-8bdb-0256180d9079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:17.530725 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc875bb-168d-4310-8bdb-0256180d9079 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:17.540719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 184/368] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:17.546524 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc875bb-168d-4310-8bdb-0256180d9079 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:17.604271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:17.642668 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfccbbf-3f20-48ba-af94-3715daab93d4 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:17.661945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:17.688487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc875bb-168d-4310-8bdb-0256180d9079 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:17.690294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f121d-d0f3-49e3-bd59-9b27c8b5798c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:17.690975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f121d-d0f3-49e3-bd59-9b27c8b5798c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:17.691257 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f121d-d0f3-49e3-bd59-9b27c8b5798c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:17.691529 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f121d-d0f3-49e3-bd59-9b27c8b5798c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:17.707999 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f121d-d0f3-49e3-bd59-9b27c8b5798c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:17.716367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-e99f121d-d0f3-49e3-bd59-9b27c8b5798c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:09:17.716367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:17.716367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:17.716367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:17.716367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:17.716367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:17.716367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:17.716367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:17.716367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:17.716367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:17.716367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:17.716367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:17.716367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:17.716367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:17.716367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:17.716367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:17.717313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:17.717313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:17.717313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:17.717313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:17.717313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:17.717313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:17.717313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:17.717313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:17.717313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:17.717313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:17.717313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:17.717313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:17.717313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:17.717313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:17.717313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:17.717313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:17.718094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:17.718094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:17.718094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:17.718094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:17.718094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:17.718094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:17.718094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:17.718094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:17.718094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:17.718094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:17.718094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:09:17.718094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:17.718094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 185/369] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:09:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:17.718094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f121d-d0f3-49e3-bd59-9b27c8b5798c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:17.738054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39714532-6726-4859-8981-28f6abb50859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:17.738674 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39714532-6726-4859-8981-28f6abb50859 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:17.738959 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39714532-6726-4859-8981-28f6abb50859 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:17.739233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39714532-6726-4859-8981-28f6abb50859 None admin] PATH_INFO: `/v3/groups/9394c6da936942d7814fb34ce09df006` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:17.740404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39714532-6726-4859-8981-28f6abb50859 None admin] RBAC: Authorizing `identity:get_group(group_id=9394c6da936942d7814fb34ce09df006)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:17.742420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39714532-6726-4859-8981-28f6abb50859 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:17.744062 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 185/370] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 12:09:17 2022] GET /identity/v3/groups/9394c6da936942d7814fb34ce09df006 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:17.744905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39714532-6726-4859-8981-28f6abb50859 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:17.848292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f121d-d0f3-49e3-bd59-9b27c8b5798c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:17.855702 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39714532-6726-4859-8981-28f6abb50859 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:17.899821 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7053b-c2c4-4019-88b3-42b2bf79073b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:17.900491 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7053b-c2c4-4019-88b3-42b2bf79073b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:17.900774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7053b-c2c4-4019-88b3-42b2bf79073b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:17.901044 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7053b-c2c4-4019-88b3-42b2bf79073b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:17.910932 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:17.913917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652743aa-082a-42a7-9959-654200855c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:17.914850 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652743aa-082a-42a7-9959-654200855c4f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:17.915268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652743aa-082a-42a7-9959-654200855c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:17.915747 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652743aa-082a-42a7-9959-654200855c4f None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:17.916549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652743aa-082a-42a7-9959-654200855c4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:17.917118 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652743aa-082a-42a7-9959-654200855c4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:17.920274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652743aa-082a-42a7-9959-654200855c4f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:17.932902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 186/371] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:17.934610 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652743aa-082a-42a7-9959-654200855c4f None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:17.955318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:17.985724 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7053b-c2c4-4019-88b3-42b2bf79073b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:17.987550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7053b-c2c4-4019-88b3-42b2bf79073b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:18.068800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652743aa-082a-42a7-9959-654200855c4f None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:18.110281 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376adfb-575f-40c3-a471-a6817a7813a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:18.111394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376adfb-575f-40c3-a471-a6817a7813a3 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:18.111706 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376adfb-575f-40c3-a471-a6817a7813a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:18.111980 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376adfb-575f-40c3-a471-a6817a7813a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:18.112728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376adfb-575f-40c3-a471-a6817a7813a3 None admin] RBAC: Adding query filter params (group.id=9394c6da936942d7814fb34ce09df006, role.id=2201e005d5ef449896a0e68a940678ec, scope.project.id=81a7733885564252af45b01ea11bfed2) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:18.113013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376adfb-575f-40c3-a471-a6817a7813a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:18.115061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376adfb-575f-40c3-a471-a6817a7813a3 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:18.116830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 186/372] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:18.118705 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7053b-c2c4-4019-88b3-42b2bf79073b None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:18.121663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 187/373] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 12:09:17 2022] GET /identity/v3/role_assignments?group.id=9394c6da936942d7814fb34ce09df006&scope.project.id=81a7733885564252af45b01ea11bfed2&role.id=2201e005d5ef449896a0e68a940678ec => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:18.123241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376adfb-575f-40c3-a471-a6817a7813a3 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:18.177632 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:18.220801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:18.258028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376adfb-575f-40c3-a471-a6817a7813a3 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:18.271367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7053b-c2c4-4019-88b3-42b2bf79073b None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:18.307613 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3e0fa-5286-45c8-9cef-015d76ead843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:18.308271 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3e0fa-5286-45c8-9cef-015d76ead843 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:18.308538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3e0fa-5286-45c8-9cef-015d76ead843 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:18.308812 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3e0fa-5286-45c8-9cef-015d76ead843 None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:18.309974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3e0fa-5286-45c8-9cef-015d76ead843 None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:18.311600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3e0fa-5286-45c8-9cef-015d76ead843 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:18.312252 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91170e-9b86-4b57-b79b-7fb1a3d3a181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:18.313077 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91170e-9b86-4b57-b79b-7fb1a3d3a181 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:18.313566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91170e-9b86-4b57-b79b-7fb1a3d3a181 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:18.314192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91170e-9b86-4b57-b79b-7fb1a3d3a181 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:18.314383 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 188/374] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:18 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:18.319161 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3e0fa-5286-45c8-9cef-015d76ead843 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:18.431868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91170e-9b86-4b57-b79b-7fb1a3d3a181 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:18.433532 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91170e-9b86-4b57-b79b-7fb1a3d3a181 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:18.499453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:18.522914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3e0fa-5286-45c8-9cef-015d76ead843 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:18.560705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5145c5f-bfc5-4da3-b82f-c7bc35857d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:18.561458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5145c5f-bfc5-4da3-b82f-c7bc35857d1a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:18.561739 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5145c5f-bfc5-4da3-b82f-c7bc35857d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:18.562014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5145c5f-bfc5-4da3-b82f-c7bc35857d1a None admin] PATH_INFO: `/v3/groups/9394c6da936942d7814fb34ce09df006` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:18.566141 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5145c5f-bfc5-4da3-b82f-c7bc35857d1a None admin] RBAC: Authorizing `identity:get_group(group_id=9394c6da936942d7814fb34ce09df006)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:18.567607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5145c5f-bfc5-4da3-b82f-c7bc35857d1a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:18.569753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 189/375] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 12:09:18 2022] GET /identity/v3/groups/9394c6da936942d7814fb34ce09df006 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:18.576031 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5145c5f-bfc5-4da3-b82f-c7bc35857d1a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:18.593161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 187/376] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:18.599060 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91170e-9b86-4b57-b79b-7fb1a3d3a181 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:18.608569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:18.759276 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5145c5f-bfc5-4da3-b82f-c7bc35857d1a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:18.799837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e1245-eb58-44d4-a187-235b24fd2b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:18.800464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e1245-eb58-44d4-a187-235b24fd2b6c None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:18.800749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e1245-eb58-44d4-a187-235b24fd2b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:18.801030 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e1245-eb58-44d4-a187-235b24fd2b6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:18.801807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e1245-eb58-44d4-a187-235b24fd2b6c None admin] RBAC: Adding query filter params (group.id=9394c6da936942d7814fb34ce09df006, role.id=8e9576d16f194938b86a1d1e087895bd, scope.project.id=81a7733885564252af45b01ea11bfed2) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:18.802121 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e1245-eb58-44d4-a187-235b24fd2b6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:18.803681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e1245-eb58-44d4-a187-235b24fd2b6c None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:18.810530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 190/377] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 12:09:18 2022] GET /identity/v3/role_assignments?group.id=9394c6da936942d7814fb34ce09df006&scope.project.id=81a7733885564252af45b01ea11bfed2&role.id=8e9576d16f194938b86a1d1e087895bd => generated 260 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:18.835543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91170e-9b86-4b57-b79b-7fb1a3d3a181 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:18.896769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51deccf-3d40-4921-83b2-bf6a63add4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:18.897491 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51deccf-3d40-4921-83b2-bf6a63add4fa None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:18.897776 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51deccf-3d40-4921-83b2-bf6a63add4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:18.898048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51deccf-3d40-4921-83b2-bf6a63add4fa None admin] PATH_INFO: `/v3/projects/2fc00be567cf4966a588e131d48e5a69` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:18.899275 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51deccf-3d40-4921-83b2-bf6a63add4fa None admin] RBAC: Authorizing `identity:get_project(project_id=2fc00be567cf4966a588e131d48e5a69)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:18.900740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51deccf-3d40-4921-83b2-bf6a63add4fa None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:18.905263 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 188/378] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:18 2022] GET /identity/v3/projects/2fc00be567cf4966a588e131d48e5a69 => generated 344 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:18.908537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e1245-eb58-44d4-a187-235b24fd2b6c None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:18.987391 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c246428-1868-4f9f-8748-11eebed98355 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:18.987749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c246428-1868-4f9f-8748-11eebed98355 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:18.987938 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c246428-1868-4f9f-8748-11eebed98355 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:18.995364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 189/379] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:09:19.001163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:19.003162 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6fe64-aa4e-4421-aa9b-2bdebe8fa90f None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:19.003447 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6fe64-aa4e-4421-aa9b-2bdebe8fa90f None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:19.003718 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6fe64-aa4e-4421-aa9b-2bdebe8fa90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:19.026796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:19.029942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e1245-eb58-44d4-a187-235b24fd2b6c None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:19.040074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-58d6fe64-aa4e-4421-aa9b-2bdebe8fa90f None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:19.041323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-58d6fe64-aa4e-4421-aa9b-2bdebe8fa90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:19.093862 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44af42-5f81-48a9-a92f-d02df4218819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:19.094422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44af42-5f81-48a9-a92f-d02df4218819 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:19.094776 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44af42-5f81-48a9-a92f-d02df4218819 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:19.094960 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44af42-5f81-48a9-a92f-d02df4218819 None admin] PATH_INFO: `/v3/groups/24e4eeee922c4b9b8d6db02bd993416f` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:19.104440 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44af42-5f81-48a9-a92f-d02df4218819 None admin] RBAC: Authorizing `identity:get_group(group_id=24e4eeee922c4b9b8d6db02bd993416f)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:19.105936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44af42-5f81-48a9-a92f-d02df4218819 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:19.108813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 191/380] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 12:09:18 2022] GET /identity/v3/groups/24e4eeee922c4b9b8d6db02bd993416f => generated 229 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:19.115073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44af42-5f81-48a9-a92f-d02df4218819 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:19.132049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 190/381] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:19.230575 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44af42-5f81-48a9-a92f-d02df4218819 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:19.272340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bd132-8187-4379-b669-5d9b63771751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:19.272962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bd132-8187-4379-b669-5d9b63771751 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:19.273276 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bd132-8187-4379-b669-5d9b63771751 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:19.273554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bd132-8187-4379-b669-5d9b63771751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:19.274347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bd132-8187-4379-b669-5d9b63771751 None admin] RBAC: Adding query filter params (group.id=24e4eeee922c4b9b8d6db02bd993416f, role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=2fc00be567cf4966a588e131d48e5a69) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:19.274668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bd132-8187-4379-b669-5d9b63771751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:19.276115 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bd132-8187-4379-b669-5d9b63771751 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:19.289235 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 192/382] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 12:09:19 2022] GET /identity/v3/role_assignments?group.id=24e4eeee922c4b9b8d6db02bd993416f&scope.project.id=2fc00be567cf4966a588e131d48e5a69&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:19.495979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6fe64-aa4e-4421-aa9b-2bdebe8fa90f None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:19.614762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6fe64-aa4e-4421-aa9b-2bdebe8fa90f None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:19.651842 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a3303-cc72-46c4-90fe-e96ab448adf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:19.652826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a3303-cc72-46c4-90fe-e96ab448adf9 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:19.653339 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a3303-cc72-46c4-90fe-e96ab448adf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:19.653864 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a3303-cc72-46c4-90fe-e96ab448adf9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:19.665071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a3303-cc72-46c4-90fe-e96ab448adf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:19.671388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:19.672870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-ad3a3303-cc72-46c4-90fe-e96ab448adf9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:19.672870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:19.672870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:19.672870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:19.672870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:19.672870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:19.672870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:19.672870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:19.672870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:19.672870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:19.672870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:19.672870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:19.672870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:19.672870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:19.672870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:19.672870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:19.674048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:19.674048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:19.674048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:19.674048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:19.674048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:19.674048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:19.674048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:19.674048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:19.674048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:19.674048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:19.674048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:19.674048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:19.674048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:19.674048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:19.674048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:19.674048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:19.674887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:19.674887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:19.674887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:19.674887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:19.674887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:19.674887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:19.674887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:19.674887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:19.674887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:19.674887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:19.674887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:19.674887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:19.677201 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 191/383] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:19.683084 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bd132-8187-4379-b669-5d9b63771751 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:19.689635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:19.820472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bd132-8187-4379-b669-5d9b63771751 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:19.864729 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b50eb2-12db-42a2-8af4-835ae827ff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:19.865417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b50eb2-12db-42a2-8af4-835ae827ff7b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:19.865691 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b50eb2-12db-42a2-8af4-835ae827ff7b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:19.865971 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b50eb2-12db-42a2-8af4-835ae827ff7b None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:19.866642 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b50eb2-12db-42a2-8af4-835ae827ff7b None admin] RBAC: Adding query filter params (name=member) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:19.866944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b50eb2-12db-42a2-8af4-835ae827ff7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:19.868330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b50eb2-12db-42a2-8af4-835ae827ff7b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:19.881958 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 193/384] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:09:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:19.883652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a3303-cc72-46c4-90fe-e96ab448adf9 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:20.017415 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a3303-cc72-46c4-90fe-e96ab448adf9 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:20.060855 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4473c-1bb1-45e9-9281-a889e56e8793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:20.061525 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4473c-1bb1-45e9-9281-a889e56e8793 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:20.061803 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4473c-1bb1-45e9-9281-a889e56e8793 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:20.062073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4473c-1bb1-45e9-9281-a889e56e8793 None admin] PATH_INFO: `/v3/groups/9394c6da936942d7814fb34ce09df006` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:20.065805 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c4473c-1bb1-45e9-9281-a889e56e8793 None admin] RBAC: Authorizing `identity:get_group(group_id=9394c6da936942d7814fb34ce09df006)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:20.067276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c4473c-1bb1-45e9-9281-a889e56e8793 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:20.069185 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 192/385] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 12:09:19 2022] GET /identity/v3/groups/9394c6da936942d7814fb34ce09df006 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:20.075679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b50eb2-12db-42a2-8af4-835ae827ff7b None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:20.181287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:20.226515 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b50eb2-12db-42a2-8af4-835ae827ff7b None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:20.233539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807d1ea-50fa-411b-8b5e-cb59fde0d456 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:20.233826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807d1ea-50fa-411b-8b5e-cb59fde0d456 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:20.234102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807d1ea-50fa-411b-8b5e-cb59fde0d456 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:20.235563 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 193/386] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:09:20.250487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a204bbb-e358-4f1e-a8ee-44e9ebd5b3cb None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:20.250766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a204bbb-e358-4f1e-a8ee-44e9ebd5b3cb None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:20.251046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a204bbb-e358-4f1e-a8ee-44e9ebd5b3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:20.267316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbb588-552e-410d-af80-d54e2184ace3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:20.267960 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbb588-552e-410d-af80-d54e2184ace3 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:20.268240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbb588-552e-410d-af80-d54e2184ace3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:20.268518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbb588-552e-410d-af80-d54e2184ace3 None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:20.270844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbb588-552e-410d-af80-d54e2184ace3 None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:20.272291 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbb588-552e-410d-af80-d54e2184ace3 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:20.274398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 194/387] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:20 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:20.279539 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbb588-552e-410d-af80-d54e2184ace3 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:20.289185 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-2a204bbb-e358-4f1e-a8ee-44e9ebd5b3cb None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:20.290347 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2a204bbb-e358-4f1e-a8ee-44e9ebd5b3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:20.388546 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbb588-552e-410d-af80-d54e2184ace3 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:20.428739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:20.429789 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d408597-8fab-4ea0-9d00-01769851624a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:20.430479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d408597-8fab-4ea0-9d00-01769851624a None admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:20.430756 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d408597-8fab-4ea0-9d00-01769851624a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:20.431029 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d408597-8fab-4ea0-9d00-01769851624a None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3/groups/9394c6da936942d7814fb34ce09df006/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:20.438288 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 194/388] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:20.439735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d408597-8fab-4ea0-9d00-01769851624a None admin] RBAC: Authorizing `identity:create_grant(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3, group_id=9394c6da936942d7814fb34ce09df006, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:20.444431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d408597-8fab-4ea0-9d00-01769851624a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:20.444431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:20.459293 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 195/389] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Sep 28 12:09:20 2022] PUT /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3/groups/9394c6da936942d7814fb34ce09df006/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:09:20.915798 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a204bbb-e358-4f1e-a8ee-44e9ebd5b3cb None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:21.030851 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a204bbb-e358-4f1e-a8ee-44e9ebd5b3cb None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:21.068759 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f756ea2-d021-4b0b-bc1b-b1373583d44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:21.069433 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f756ea2-d021-4b0b-bc1b-b1373583d44a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:21.069708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f756ea2-d021-4b0b-bc1b-b1373583d44a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:21.069977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f756ea2-d021-4b0b-bc1b-b1373583d44a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:21.084752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f756ea2-d021-4b0b-bc1b-b1373583d44a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:21.089884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-9f756ea2-d021-4b0b-bc1b-b1373583d44a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:09:21.089884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:21.089884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:21.089884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:21.089884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:21.089884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:21.089884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:21.089884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:21.089884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:21.089884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:21.089884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:21.089884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:21.089884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:21.089884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:21.089884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:21.089884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:21.090789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:21.090789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:21.090789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:21.090789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:21.090789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:21.090789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:21.090789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:21.090789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:21.090789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:21.090789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:21.090789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:21.090789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:21.090789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:21.090789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:21.090789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:21.090789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:21.091601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:21.091601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:21.091601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:21.091601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:21.091601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:21.091601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:21.091601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:21.091601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:21.091601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:21.091601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:21.091601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:09:21.091601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:21.091601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 195/390] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:09:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:21.098143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d408597-8fab-4ea0-9d00-01769851624a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:21.159153 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efbd4f-8688-47c7-bf8b-6b4fffc6ea4c None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:21.159479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efbd4f-8688-47c7-bf8b-6b4fffc6ea4c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:21.159699 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efbd4f-8688-47c7-bf8b-6b4fffc6ea4c None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:21.161372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 196/391] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:21.166370 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6001ea7-ed4c-4a45-bc2e-de91f1e7d6ba None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:21.166892 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6001ea7-ed4c-4a45-bc2e-de91f1e7d6ba None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:21.167386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6001ea7-ed4c-4a45-bc2e-de91f1e7d6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:21.199788 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-d6001ea7-ed4c-4a45-bc2e-de91f1e7d6ba None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:21.200973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d6001ea7-ed4c-4a45-bc2e-de91f1e7d6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:21.244326 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d408597-8fab-4ea0-9d00-01769851624a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:21.281241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7ec71-7abc-43b8-af1c-6b274d428a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:21.282234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7ec71-7abc-43b8-af1c-6b274d428a93 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:21.282512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7ec71-7abc-43b8-af1c-6b274d428a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:21.282810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7ec71-7abc-43b8-af1c-6b274d428a93 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:21.283627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7ec71-7abc-43b8-af1c-6b274d428a93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:21.283927 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7ec71-7abc-43b8-af1c-6b274d428a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:21.285306 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b7ec71-7abc-43b8-af1c-6b274d428a93 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:21.293713 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 196/392] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:21.299863 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7ec71-7abc-43b8-af1c-6b274d428a93 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:21.354853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:21.369519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:21.408857 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 197/393] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:09:21.447329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7ec71-7abc-43b8-af1c-6b274d428a93 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:21.520939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b749f1-bc5e-4de9-9c36-78be357c3c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:21.521879 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b749f1-bc5e-4de9-9c36-78be357c3c8e None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:21.522329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b749f1-bc5e-4de9-9c36-78be357c3c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:21.522754 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b749f1-bc5e-4de9-9c36-78be357c3c8e None admin] PATH_INFO: `/v3/groups/9394c6da936942d7814fb34ce09df006` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:21.525922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b749f1-bc5e-4de9-9c36-78be357c3c8e None admin] RBAC: Authorizing `identity:get_group(group_id=9394c6da936942d7814fb34ce09df006)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:21.527520 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b749f1-bc5e-4de9-9c36-78be357c3c8e None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:21.529271 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 197/394] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 12:09:21 2022] GET /identity/v3/groups/9394c6da936942d7814fb34ce09df006 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:21.535641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6001ea7-ed4c-4a45-bc2e-de91f1e7d6ba None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:21.616783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304c61f-aff4-41d5-b00d-47c9d12e9f6c None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:21.617092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304c61f-aff4-41d5-b00d-47c9d12e9f6c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:21.617366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304c61f-aff4-41d5-b00d-47c9d12e9f6c None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:21.618697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 198/395] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:21.625195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad263ece-3417-4922-b705-31d0592e5f39 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:21.626014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad263ece-3417-4922-b705-31d0592e5f39 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:21.626604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad263ece-3417-4922-b705-31d0592e5f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:21.646141 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-ad263ece-3417-4922-b705-31d0592e5f39 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:21.647664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-ad263ece-3417-4922-b705-31d0592e5f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:21.666687 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6001ea7-ed4c-4a45-bc2e-de91f1e7d6ba None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:21.704238 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92db5c8-26e6-48bb-9507-d45f3f60c543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:21.705635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92db5c8-26e6-48bb-9507-d45f3f60c543 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:21.705925 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92db5c8-26e6-48bb-9507-d45f3f60c543 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:21.706196 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92db5c8-26e6-48bb-9507-d45f3f60c543 None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:21.707334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92db5c8-26e6-48bb-9507-d45f3f60c543 None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:21.708773 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92db5c8-26e6-48bb-9507-d45f3f60c543 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:21.710850 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 198/396] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:21 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:21.716627 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92db5c8-26e6-48bb-9507-d45f3f60c543 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:21.763214 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 199/397] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:09:21.806599 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad263ece-3417-4922-b705-31d0592e5f39 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:21.833441 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:21.887224 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:21.936576 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92db5c8-26e6-48bb-9507-d45f3f60c543 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:21.958595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad263ece-3417-4922-b705-31d0592e5f39 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:22.007379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6fdb7-2544-44b9-850e-0dff50abef3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:22.008048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6fdb7-2544-44b9-850e-0dff50abef3c None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:22.008333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6fdb7-2544-44b9-850e-0dff50abef3c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:22.008610 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6fdb7-2544-44b9-850e-0dff50abef3c None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3/groups/9394c6da936942d7814fb34ce09df006/roles/8e9576d16f194938b86a1d1e087895bd` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:22.015683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6fdb7-2544-44b9-850e-0dff50abef3c None admin] RBAC: Authorizing `identity:create_grant(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3, group_id=9394c6da936942d7814fb34ce09df006, role_id=8e9576d16f194938b86a1d1e087895bd)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:22.017228 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6fdb7-2544-44b9-850e-0dff50abef3c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:22.026230 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe50dc-0f55-4df1-b6eb-cd04a7b1a4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:22.026929 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe50dc-0f55-4df1-b6eb-cd04a7b1a4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:22.027209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe50dc-0f55-4df1-b6eb-cd04a7b1a4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:22.027503 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe50dc-0f55-4df1-b6eb-cd04a7b1a4f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:22.038269 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 199/398] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Sep 28 12:09:21 2022] PUT /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3/groups/9394c6da936942d7814fb34ce09df006/roles/8e9576d16f194938b86a1d1e087895bd => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:09:22.051295 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe50dc-0f55-4df1-b6eb-cd04a7b1a4f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:22.056285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-96fe50dc-0f55-4df1-b6eb-cd04a7b1a4f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:22.056285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:22.056285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:22.056285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:22.056285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:22.056285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:22.056285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:22.056285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:22.056285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:22.056285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:22.056285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:22.056285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:22.056285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:22.056285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:22.056285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:22.056285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:22.057196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:22.057196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:22.057196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:22.057196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:22.057196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:22.057196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:22.057196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:22.057196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:22.057196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:22.057196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:22.057196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:22.057196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:22.057196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:22.057196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:22.057196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:22.057196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:22.058035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:22.058035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:22.058035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:22.058035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:22.058035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:22.058035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:22.058035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:22.058035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:22.058035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:22.058035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:22.058035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:22.058035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:22.058035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 200/399] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:22.066005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6fdb7-2544-44b9-850e-0dff50abef3c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:22.066780 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe50dc-0f55-4df1-b6eb-cd04a7b1a4f8 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:22.210261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6fdb7-2544-44b9-850e-0dff50abef3c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:22.249514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:22.273118 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe50dc-0f55-4df1-b6eb-cd04a7b1a4f8 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:22.288712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbb762-c34b-42be-8bcf-452f54125f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:22.289416 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbb762-c34b-42be-8bcf-452f54125f83 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:22.289697 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbb762-c34b-42be-8bcf-452f54125f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:22.289943 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:22.290045 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbb762-c34b-42be-8bcf-452f54125f83 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:22.290736 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbb762-c34b-42be-8bcf-452f54125f83 None admin] RBAC: Adding query filter params (name=member) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:22.291083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbb762-c34b-42be-8bcf-452f54125f83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:22.292659 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbb762-c34b-42be-8bcf-452f54125f83 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:22.301900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 200/400] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:09:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:22.318105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbb762-c34b-42be-8bcf-452f54125f83 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:22.318523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62db017-6374-43fa-80c0-9dabde3ac913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:22.319178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62db017-6374-43fa-80c0-9dabde3ac913 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:22.319455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62db017-6374-43fa-80c0-9dabde3ac913 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:22.319725 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62db017-6374-43fa-80c0-9dabde3ac913 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:22.331872 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62db017-6374-43fa-80c0-9dabde3ac913 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:22.336887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-b62db017-6374-43fa-80c0-9dabde3ac913 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:09:22.336887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:22.336887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:22.336887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:22.336887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:22.336887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:22.336887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:22.336887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:22.336887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:22.336887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:22.336887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:22.336887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:22.336887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:22.336887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:22.336887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:22.336887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:22.338012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:22.338012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:22.338012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:22.338012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:22.338012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:22.338012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:22.338012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:22.338012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:22.338012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:22.338012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:22.338012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:22.338012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:22.338012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:22.338012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:22.338012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:22.338012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:22.338970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:22.338970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:22.338970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:22.338970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:22.338970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:22.338970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:22.338970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:22.338970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:22.338970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:22.338970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:22.338970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:09:22.338970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:22.338970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 201/401] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:09:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:22.347006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525edd4-fe3c-4832-89d6-2a5aab2ff9b1 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:22.347463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525edd4-fe3c-4832-89d6-2a5aab2ff9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:22.347890 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525edd4-fe3c-4832-89d6-2a5aab2ff9b1 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:22.350229 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 202/402] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:22.351069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525edd4-fe3c-4832-89d6-2a5aab2ff9b1 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:22.441678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbb762-c34b-42be-8bcf-452f54125f83 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:22.472183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525edd4-fe3c-4832-89d6-2a5aab2ff9b1 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:22.478182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b60e55-42a7-446f-9b9a-c3d5847a572d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:22.479018 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b60e55-42a7-446f-9b9a-c3d5847a572d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:22.479435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b60e55-42a7-446f-9b9a-c3d5847a572d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:22.479840 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b60e55-42a7-446f-9b9a-c3d5847a572d None admin] PATH_INFO: `/v3/groups/9394c6da936942d7814fb34ce09df006` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:22.481549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b60e55-42a7-446f-9b9a-c3d5847a572d None admin] RBAC: Authorizing `identity:get_group(group_id=9394c6da936942d7814fb34ce09df006)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:22.483271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b60e55-42a7-446f-9b9a-c3d5847a572d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:22.489423 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 201/403] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 12:09:22 2022] GET /identity/v3/groups/9394c6da936942d7814fb34ce09df006 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:22.490952 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e4cb9-476d-44b0-8163-e29f26fe20c7 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:22.491453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e4cb9-476d-44b0-8163-e29f26fe20c7 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:22.491902 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e4cb9-476d-44b0-8163-e29f26fe20c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:22.511491 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fc284-2c73-4c68-a0f1-37508dafbcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:22.512260 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fc284-2c73-4c68-a0f1-37508dafbcbc None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:22.512732 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fc284-2c73-4c68-a0f1-37508dafbcbc None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:22.513098 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fc284-2c73-4c68-a0f1-37508dafbcbc None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:22.513762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-037e4cb9-476d-44b0-8163-e29f26fe20c7 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:22.513968 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fc284-2c73-4c68-a0f1-37508dafbcbc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:22.514381 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fc284-2c73-4c68-a0f1-37508dafbcbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:22.515422 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-037e4cb9-476d-44b0-8163-e29f26fe20c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:22.515927 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fc284-2c73-4c68-a0f1-37508dafbcbc None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:22.524807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 203/404] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:22.530285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fc284-2c73-4c68-a0f1-37508dafbcbc None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:22.565661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:22.569919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:22.593514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 202/405] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:22.594716 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e4cb9-476d-44b0-8163-e29f26fe20c7 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:22.651685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fc284-2c73-4c68-a0f1-37508dafbcbc None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:22.689812 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278096d-d3f5-4f1d-9119-e01c62231431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:22.690673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278096d-d3f5-4f1d-9119-e01c62231431 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:22.691288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278096d-d3f5-4f1d-9119-e01c62231431 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:22.691680 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278096d-d3f5-4f1d-9119-e01c62231431 None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:22.693821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278096d-d3f5-4f1d-9119-e01c62231431 None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:22.698641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278096d-d3f5-4f1d-9119-e01c62231431 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:22.701718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 204/406] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:22 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:22.707156 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278096d-d3f5-4f1d-9119-e01c62231431 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:22.757317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e4cb9-476d-44b0-8163-e29f26fe20c7 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:22.832511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52b649-7fe4-45a4-9606-d7bebada0675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:22.833372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52b649-7fe4-45a4-9606-d7bebada0675 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:22.833753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52b649-7fe4-45a4-9606-d7bebada0675 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:22.834149 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52b649-7fe4-45a4-9606-d7bebada0675 None admin] PATH_INFO: `/v3/groups/9394c6da936942d7814fb34ce09df006` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:22.835490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52b649-7fe4-45a4-9606-d7bebada0675 None admin] RBAC: Authorizing `identity:get_group(group_id=9394c6da936942d7814fb34ce09df006)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:22.837042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52b649-7fe4-45a4-9606-d7bebada0675 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:22.839530 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 203/407] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 12:09:22 2022] GET /identity/v3/groups/9394c6da936942d7814fb34ce09df006 => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:22.850454 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52b649-7fe4-45a4-9606-d7bebada0675 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:22.880719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278096d-d3f5-4f1d-9119-e01c62231431 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:22.909529 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:22.917264 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:22.924984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be6e74-028a-4fa1-a413-9d37d0572443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:22.925797 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be6e74-028a-4fa1-a413-9d37d0572443 None admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:22.926182 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be6e74-028a-4fa1-a413-9d37d0572443 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:22.926576 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be6e74-028a-4fa1-a413-9d37d0572443 None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2/groups/9394c6da936942d7814fb34ce09df006/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:22.928775 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be6e74-028a-4fa1-a413-9d37d0572443 None admin] RBAC: Authorizing `identity:create_grant(project_id=81a7733885564252af45b01ea11bfed2, group_id=9394c6da936942d7814fb34ce09df006, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:22.930679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be6e74-028a-4fa1-a413-9d37d0572443 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:22.953251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 205/408] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Sep 28 12:09:22 2022] PUT /identity/v3/projects/81a7733885564252af45b01ea11bfed2/groups/9394c6da936942d7814fb34ce09df006/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:09:22.955828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be6e74-028a-4fa1-a413-9d37d0572443 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:23.031456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52b649-7fe4-45a4-9606-d7bebada0675 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:23.086122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d55e36-f020-473a-9f44-fb38ac7a7f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:23.086864 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d55e36-f020-473a-9f44-fb38ac7a7f95 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:23.087145 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d55e36-f020-473a-9f44-fb38ac7a7f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:23.087423 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d55e36-f020-473a-9f44-fb38ac7a7f95 None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:23.088655 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d55e36-f020-473a-9f44-fb38ac7a7f95 None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:23.090193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d55e36-f020-473a-9f44-fb38ac7a7f95 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:23.092256 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 204/409] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:22 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:23.098844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d55e36-f020-473a-9f44-fb38ac7a7f95 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:23.173495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be6e74-028a-4fa1-a413-9d37d0572443 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:23.219551 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ec169-0b31-44db-876d-49b92f92159f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:23.220874 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ec169-0b31-44db-876d-49b92f92159f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:23.221269 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ec169-0b31-44db-876d-49b92f92159f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:23.221547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ec169-0b31-44db-876d-49b92f92159f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:23.236603 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ec169-0b31-44db-876d-49b92f92159f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:23.243083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-a89ec169-0b31-44db-876d-49b92f92159f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:09:23.243083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:23.243083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:23.243083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:23.243083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:23.243083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:23.243083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:23.243083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:23.243083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:23.243083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:23.243083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:23.243083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:23.243083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:23.243083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:23.243083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:23.243083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:23.243992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:23.243992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:23.243992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:23.243992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:23.243992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:23.243992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:23.243992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:23.243992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:23.243992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:23.243992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:23.243992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:23.243992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:23.243992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:23.243992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:23.243992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:23.243992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:23.244819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:23.244819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:23.244819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:23.244819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:23.244819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:23.244819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:23.244819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:23.244819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:23.244819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:23.244819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:23.244819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:09:23.244819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:23.244819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 206/410] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:09:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:23.250663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ec169-0b31-44db-876d-49b92f92159f None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:23.260531 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d55e36-f020-473a-9f44-fb38ac7a7f95 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:23.272486 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:23.287229 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:23.308251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ada92e-6583-49c4-aa03-9fb94536c24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:23.309339 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ada92e-6583-49c4-aa03-9fb94536c24b None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:23.309821 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ada92e-6583-49c4-aa03-9fb94536c24b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:23.310386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ada92e-6583-49c4-aa03-9fb94536c24b None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2/groups/9394c6da936942d7814fb34ce09df006/roles/8e9576d16f194938b86a1d1e087895bd` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:23.312837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ada92e-6583-49c4-aa03-9fb94536c24b None admin] RBAC: Authorizing `identity:create_grant(project_id=81a7733885564252af45b01ea11bfed2, group_id=9394c6da936942d7814fb34ce09df006, role_id=8e9576d16f194938b86a1d1e087895bd)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:23.314564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ada92e-6583-49c4-aa03-9fb94536c24b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:23.337168 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 205/411] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Sep 28 12:09:23 2022] PUT /identity/v3/projects/81a7733885564252af45b01ea11bfed2/groups/9394c6da936942d7814fb34ce09df006/roles/8e9576d16f194938b86a1d1e087895bd => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:09:23.337168 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223a053-cfb3-4ccd-8d7d-8ec86f12a760 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:23.337168 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223a053-cfb3-4ccd-8d7d-8ec86f12a760 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:23.337168 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223a053-cfb3-4ccd-8d7d-8ec86f12a760 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:23.337168 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 206/412] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:23.342887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3870f31-7c22-4604-b5f8-feedc3da8288 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:23.343301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3870f31-7c22-4604-b5f8-feedc3da8288 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:23.343670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3870f31-7c22-4604-b5f8-feedc3da8288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:23.385102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-d3870f31-7c22-4604-b5f8-feedc3da8288 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:23.386483 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d3870f31-7c22-4604-b5f8-feedc3da8288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:23.452034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ec169-0b31-44db-876d-49b92f92159f None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:23.483923 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 207/413] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:23.534840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2346cac-8d82-4831-b415-fca2d57c5c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:23.535805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2346cac-8d82-4831-b415-fca2d57c5c7f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:23.536298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2346cac-8d82-4831-b415-fca2d57c5c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:23.536792 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2346cac-8d82-4831-b415-fca2d57c5c7f None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:23.537749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2346cac-8d82-4831-b415-fca2d57c5c7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:23.538254 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2346cac-8d82-4831-b415-fca2d57c5c7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:23.539902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2346cac-8d82-4831-b415-fca2d57c5c7f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:23.553243 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 207/414] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:23.556255 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3870f31-7c22-4604-b5f8-feedc3da8288 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:23.708656 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:23.712721 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3870f31-7c22-4604-b5f8-feedc3da8288 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:23.763174 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebcf9a-c04d-4707-9243-1cceae625f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:23.763818 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebcf9a-c04d-4707-9243-1cceae625f8a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:23.764104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebcf9a-c04d-4707-9243-1cceae625f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:23.764383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebcf9a-c04d-4707-9243-1cceae625f8a None admin] PATH_INFO: `/v3/groups/24e4eeee922c4b9b8d6db02bd993416f` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:23.765549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eebcf9a-c04d-4707-9243-1cceae625f8a None admin] RBAC: Authorizing `identity:get_group(group_id=24e4eeee922c4b9b8d6db02bd993416f)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:23.767537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eebcf9a-c04d-4707-9243-1cceae625f8a None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:23.769481 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 208/415] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 12:09:23 2022] GET /identity/v3/groups/24e4eeee922c4b9b8d6db02bd993416f => generated 229 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:23.775055 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2346cac-8d82-4831-b415-fca2d57c5c7f None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:23.781258 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebcf9a-c04d-4707-9243-1cceae625f8a None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:23.949276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebcf9a-c04d-4707-9243-1cceae625f8a None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:23.953501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:23.982789 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2346cac-8d82-4831-b415-fca2d57c5c7f None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:24.012294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bd6b3-d668-4083-80d8-0be6f30e872d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:24.013196 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bd6b3-d668-4083-80d8-0be6f30e872d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:24.013648 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bd6b3-d668-4083-80d8-0be6f30e872d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:24.014054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6bd6b3-d668-4083-80d8-0be6f30e872d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:24.035340 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6bd6b3-d668-4083-80d8-0be6f30e872d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:24.044097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-9f6bd6b3-d668-4083-80d8-0be6f30e872d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:24.044097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:24.044097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:24.044097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:24.044097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:24.044097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:24.044097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:24.044097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:24.044097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:24.044097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:24.044097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:24.044097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:24.044097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:24.044097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:24.044097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:24.044097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:24.045204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:24.045204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:24.045204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:24.045204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:24.045204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:24.045204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:24.045204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:24.045204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:24.045204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:24.045204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:24.045204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:24.045204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:24.045204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:24.045204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:24.045204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:24.045204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:24.046166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:24.046166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:24.046166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:24.046166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:24.046166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:24.046166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:24.046166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:24.046166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:24.046166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:24.046166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:24.046166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:24.046166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:24.057619 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ec5a7-be04-4837-92a2-f5180bf6fbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:24.058505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ec5a7-be04-4837-92a2-f5180bf6fbc6 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:24.058925 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ec5a7-be04-4837-92a2-f5180bf6fbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:24.059088 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 209/416] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:24.059490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ec5a7-be04-4837-92a2-f5180bf6fbc6 None admin] PATH_INFO: `/v3/projects/2fc00be567cf4966a588e131d48e5a69` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:24.060589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bd6b3-d668-4083-80d8-0be6f30e872d None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:24.063028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ec5a7-be04-4837-92a2-f5180bf6fbc6 None admin] RBAC: Authorizing `identity:get_project(project_id=2fc00be567cf4966a588e131d48e5a69)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:24.064801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ec5a7-be04-4837-92a2-f5180bf6fbc6 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:24.069828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 208/417] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:23 2022] GET /identity/v3/projects/2fc00be567cf4966a588e131d48e5a69 => generated 344 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:24.076896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ec5a7-be04-4837-92a2-f5180bf6fbc6 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:24.123214 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:24.248717 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6bd6b3-d668-4083-80d8-0be6f30e872d None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:24.267500 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ec5a7-be04-4837-92a2-f5180bf6fbc6 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:24.318706 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed5c81-c3e5-4119-aaba-5c2961e68a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:24.319732 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed5c81-c3e5-4119-aaba-5c2961e68a4b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:24.320189 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed5c81-c3e5-4119-aaba-5c2961e68a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:24.320644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed5c81-c3e5-4119-aaba-5c2961e68a4b None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:24.321521 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed5c81-c3e5-4119-aaba-5c2961e68a4b None admin] RBAC: Adding query filter params (name=member) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:24.322011 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed5c81-c3e5-4119-aaba-5c2961e68a4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:24.325261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed5c81-c3e5-4119-aaba-5c2961e68a4b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:24.335067 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 210/418] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:09:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:24.342148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed5c81-c3e5-4119-aaba-5c2961e68a4b None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:24.350042 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ba81e-0cf2-4ff1-ab2a-e8004ea3dcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:24.350544 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ba81e-0cf2-4ff1-ab2a-e8004ea3dcd1 None admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:24.350840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ba81e-0cf2-4ff1-ab2a-e8004ea3dcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:24.351089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ba81e-0cf2-4ff1-ab2a-e8004ea3dcd1 None admin] PATH_INFO: `/v3/projects/2fc00be567cf4966a588e131d48e5a69/groups/24e4eeee922c4b9b8d6db02bd993416f/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:24.358619 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ba81e-0cf2-4ff1-ab2a-e8004ea3dcd1 None admin] RBAC: Authorizing `identity:create_grant(project_id=2fc00be567cf4966a588e131d48e5a69, group_id=24e4eeee922c4b9b8d6db02bd993416f, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:24.360471 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ba81e-0cf2-4ff1-ab2a-e8004ea3dcd1 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:24.379930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 209/419] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Sep 28 12:09:24 2022] PUT /identity/v3/projects/2fc00be567cf4966a588e131d48e5a69/groups/24e4eeee922c4b9b8d6db02bd993416f/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:09:24.456209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:24.474070 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed5c81-c3e5-4119-aaba-5c2961e68a4b None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:24.518467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae4f52-7364-4cac-8438-f451ee5e6903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:24.519002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae4f52-7364-4cac-8438-f451ee5e6903 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:24.519338 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae4f52-7364-4cac-8438-f451ee5e6903 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:24.519612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae4f52-7364-4cac-8438-f451ee5e6903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:24.606940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da323d-d742-45ae-9291-b0e241984999 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:24.607291 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da323d-d742-45ae-9291-b0e241984999 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:24.607484 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da323d-d742-45ae-9291-b0e241984999 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:24.608831 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 210/420] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:09:24.609090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae4f52-7364-4cac-8438-f451ee5e6903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:24.611291 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fae4f52-7364-4cac-8438-f451ee5e6903 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:24.614932 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dca84-aea5-422d-9f2c-edb81e9348bb None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:24.615213 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dca84-aea5-422d-9f2c-edb81e9348bb None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:24.615485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dca84-aea5-422d-9f2c-edb81e9348bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:24.644408 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-706dca84-aea5-422d-9f2c-edb81e9348bb None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:24.645658 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-706dca84-aea5-422d-9f2c-edb81e9348bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:24.656250 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:24.732048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:24.735019 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 211/421] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:09:24.761348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 211/422] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 12:09:24.767141 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706dca84-aea5-422d-9f2c-edb81e9348bb None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:24.891062 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706dca84-aea5-422d-9f2c-edb81e9348bb None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:24.928433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f65b9f-83a3-4176-b4d9-f4ab9fde9376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:24.929408 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f65b9f-83a3-4176-b4d9-f4ab9fde9376 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:24.929867 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f65b9f-83a3-4176-b4d9-f4ab9fde9376 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:24.930389 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f65b9f-83a3-4176-b4d9-f4ab9fde9376 None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:24.931709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f65b9f-83a3-4176-b4d9-f4ab9fde9376 None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:24.933360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f65b9f-83a3-4176-b4d9-f4ab9fde9376 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:24.935467 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 212/423] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:24 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:24.942218 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae4f52-7364-4cac-8438-f451ee5e6903 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:25.089697 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae4f52-7364-4cac-8438-f451ee5e6903 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:25.137588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a5f931-bade-44f1-a33f-d8594d4450cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:25.138522 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5f931-bade-44f1-a33f-d8594d4450cc None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:25.138982 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5f931-bade-44f1-a33f-d8594d4450cc None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:25.139455 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a5f931-bade-44f1-a33f-d8594d4450cc None admin] PATH_INFO: `/v3/groups/9394c6da936942d7814fb34ce09df006` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:25.140740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a5f931-bade-44f1-a33f-d8594d4450cc None admin] RBAC: Authorizing `identity:get_group(group_id=9394c6da936942d7814fb34ce09df006)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:25.143066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a5f931-bade-44f1-a33f-d8594d4450cc None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:25.144830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 212/424] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 12:09:24 2022] GET /identity/v3/groups/9394c6da936942d7814fb34ce09df006 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:25.150620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f65b9f-83a3-4176-b4d9-f4ab9fde9376 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:25.175407 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:25.215559 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a5f931-bade-44f1-a33f-d8594d4450cc None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:25.309870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:25.342164 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f65b9f-83a3-4176-b4d9-f4ab9fde9376 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:25.390759 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c4d23-c4ec-4928-9050-cf7d7e7990eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:25.391390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c4d23-c4ec-4928-9050-cf7d7e7990eb None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:25.391620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c4d23-c4ec-4928-9050-cf7d7e7990eb None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:25.392410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c4d23-c4ec-4928-9050-cf7d7e7990eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:25.393212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c4d23-c4ec-4928-9050-cf7d7e7990eb None admin] RBAC: Adding query filter params (group.id=9394c6da936942d7814fb34ce09df006, role.id=2201e005d5ef449896a0e68a940678ec, scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:25.393462 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c4d23-c4ec-4928-9050-cf7d7e7990eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:25.394875 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c4d23-c4ec-4928-9050-cf7d7e7990eb None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:25.398245 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a5f931-bade-44f1-a33f-d8594d4450cc None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:25.414427 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 213/425] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 12:09:25 2022] GET /identity/v3/role_assignments?group.id=9394c6da936942d7814fb34ce09df006&scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3&role.id=2201e005d5ef449896a0e68a940678ec => generated 611 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:25.443019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698441cc-0da7-4672-baae-7326d28f9f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:25.443737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698441cc-0da7-4672-baae-7326d28f9f8c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:25.444017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698441cc-0da7-4672-baae-7326d28f9f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:25.444347 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698441cc-0da7-4672-baae-7326d28f9f8c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:25.457930 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698441cc-0da7-4672-baae-7326d28f9f8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:25.462979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-698441cc-0da7-4672-baae-7326d28f9f8c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:09:25.462979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:25.462979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:25.462979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:25.462979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:25.462979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:25.462979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:25.462979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:25.462979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:25.462979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:25.462979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:25.462979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:25.462979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:25.462979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:25.462979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:25.462979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:25.463908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:25.463908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:25.463908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:25.463908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:25.463908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:25.463908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:25.463908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:25.463908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:25.463908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:25.463908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:25.463908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:25.463908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:25.463908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:25.463908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:25.463908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:25.463908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:25.464734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:25.464734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:25.464734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:25.464734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:25.464734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:25.464734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:25.464734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:25.464734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:25.464734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:25.464734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:25.464734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:09:25.464734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:25.464734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 213/426] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:09:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:25.469903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c4d23-c4ec-4928-9050-cf7d7e7990eb None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:25.600897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:25.635355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c4d23-c4ec-4928-9050-cf7d7e7990eb None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:25.676256 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7de55b-561c-4050-898e-8e888e13977f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:25.677116 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7de55b-561c-4050-898e-8e888e13977f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:25.677570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7de55b-561c-4050-898e-8e888e13977f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:25.678008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7de55b-561c-4050-898e-8e888e13977f None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:25.678814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7de55b-561c-4050-898e-8e888e13977f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:25.679262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7de55b-561c-4050-898e-8e888e13977f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:25.680785 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7de55b-561c-4050-898e-8e888e13977f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:25.690415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 214/427] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:25.695888 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698441cc-0da7-4672-baae-7326d28f9f8c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:25.743356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51243b3a-feda-410f-a654-70b02a446134 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:25.743873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51243b3a-feda-410f-a654-70b02a446134 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:25.744297 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51243b3a-feda-410f-a654-70b02a446134 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:25.746057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 215/428] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:25.750855 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8dc8e-b55e-439f-b3f2-5bd5143a5a52 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:25.751528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8dc8e-b55e-439f-b3f2-5bd5143a5a52 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:25.752042 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8dc8e-b55e-439f-b3f2-5bd5143a5a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:25.776331 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-72f8dc8e-b55e-439f-b3f2-5bd5143a5a52 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:25.777943 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-72f8dc8e-b55e-439f-b3f2-5bd5143a5a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:25.818342 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698441cc-0da7-4672-baae-7326d28f9f8c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:25.838594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:25.857049 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 216/429] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:25.859762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872cb12-f694-465a-9bad-9021e7cc5a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:25.859762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872cb12-f694-465a-9bad-9021e7cc5a97 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:25.859762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872cb12-f694-465a-9bad-9021e7cc5a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:25.859762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872cb12-f694-465a-9bad-9021e7cc5a97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:25.956168 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872cb12-f694-465a-9bad-9021e7cc5a97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:25.957602 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872cb12-f694-465a-9bad-9021e7cc5a97 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:26.081381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 214/430] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:26.087366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8dc8e-b55e-439f-b3f2-5bd5143a5a52 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:26.131946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2d817-17bc-4132-97dd-5bdf99ef3a69 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:26.132445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2d817-17bc-4132-97dd-5bdf99ef3a69 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:26.132544 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2d817-17bc-4132-97dd-5bdf99ef3a69 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:26.134304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 215/431] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:26.139294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64024de0-7e8f-4c02-8d70-71f1689a242e None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:26.139580 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64024de0-7e8f-4c02-8d70-71f1689a242e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:26.139849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64024de0-7e8f-4c02-8d70-71f1689a242e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:26.159146 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-64024de0-7e8f-4c02-8d70-71f1689a242e None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:26.160446 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-64024de0-7e8f-4c02-8d70-71f1689a242e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:26.174016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:26.184074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:26.230400 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8dc8e-b55e-439f-b3f2-5bd5143a5a52 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:26.244910 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 216/432] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:26.245630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64024de0-7e8f-4c02-8d70-71f1689a242e None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:26.274745 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c27ab-d1dd-481b-91f9-ce2b98a8ab28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:26.275877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c27ab-d1dd-481b-91f9-ce2b98a8ab28 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:26.276450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c27ab-d1dd-481b-91f9-ce2b98a8ab28 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:26.276942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c27ab-d1dd-481b-91f9-ce2b98a8ab28 None admin] PATH_INFO: `/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:26.278302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c27ab-d1dd-481b-91f9-ce2b98a8ab28 None admin] RBAC: Authorizing `identity:get_project(project_id=04a8a1a229ca4dd7aa955d90ed2e86c3)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:26.280767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c27ab-d1dd-481b-91f9-ce2b98a8ab28 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:26.283327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 217/433] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:26 2022] GET /identity/v3/projects/04a8a1a229ca4dd7aa955d90ed2e86c3 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:26.288369 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c27ab-d1dd-481b-91f9-ce2b98a8ab28 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:26.375175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64024de0-7e8f-4c02-8d70-71f1689a242e None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:26.399444 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c27ab-d1dd-481b-91f9-ce2b98a8ab28 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:26.427657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e497c-205b-4c52-b2fd-96a02a84fc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:26.429185 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e497c-205b-4c52-b2fd-96a02a84fc04 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:26.429667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e497c-205b-4c52-b2fd-96a02a84fc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:26.430142 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e497c-205b-4c52-b2fd-96a02a84fc04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:26.440568 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db564139-29f7-4c76-b920-cd42b0d8565b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:26.441224 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db564139-29f7-4c76-b920-cd42b0d8565b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:26.441503 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db564139-29f7-4c76-b920-cd42b0d8565b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:26.441772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db564139-29f7-4c76-b920-cd42b0d8565b None admin] PATH_INFO: `/v3/groups/9394c6da936942d7814fb34ce09df006` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:26.442895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db564139-29f7-4c76-b920-cd42b0d8565b None admin] RBAC: Authorizing `identity:get_group(group_id=9394c6da936942d7814fb34ce09df006)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:26.444110 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e497c-205b-4c52-b2fd-96a02a84fc04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:26.444597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db564139-29f7-4c76-b920-cd42b0d8565b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:26.449487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-811e497c-205b-4c52-b2fd-96a02a84fc04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:26.449487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:26.449487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:26.449487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:26.449487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:26.449487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:26.449487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:26.449487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:26.449487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:26.449487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:26.449487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:26.449487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:26.449487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:26.449487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:26.449487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:26.449487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:26.450348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:26.450348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:26.450348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:26.450348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:26.450348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:26.450348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:26.450348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:26.450348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:26.450348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:26.450348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:26.450348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:26.450348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:26.450348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:26.450348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:26.450348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:26.450348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:26.451143 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:26.451143 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:26.451143 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:26.451143 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:26.451143 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:26.451143 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:26.451143 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:26.451143 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:26.451143 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:26.451143 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:26.451143 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 12:09:26.451143 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:26.451741 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 218/434] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 12:09:26 2022] GET /identity/v3/groups/9394c6da936942d7814fb34ce09df006 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:26.452669 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db564139-29f7-4c76-b920-cd42b0d8565b None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:26.453922 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 217/435] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:26.456550 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:26.459876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e497c-205b-4c52-b2fd-96a02a84fc04 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:26.471330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:26.572277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db564139-29f7-4c76-b920-cd42b0d8565b None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:26.586766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e497c-205b-4c52-b2fd-96a02a84fc04 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:26.614285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13c8d6-8d90-4b9c-b8fb-92b0531b71ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:26.615995 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13c8d6-8d90-4b9c-b8fb-92b0531b71ed None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:26.616467 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13c8d6-8d90-4b9c-b8fb-92b0531b71ed None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:26.617107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13c8d6-8d90-4b9c-b8fb-92b0531b71ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:26.617882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13c8d6-8d90-4b9c-b8fb-92b0531b71ed None admin] RBAC: Adding query filter params (group.id=9394c6da936942d7814fb34ce09df006, role.id=8e9576d16f194938b86a1d1e087895bd, scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:26.618185 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13c8d6-8d90-4b9c-b8fb-92b0531b71ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:26.619639 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13c8d6-8d90-4b9c-b8fb-92b0531b71ed None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:26.629518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 219/436] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 12:09:26 2022] GET /identity/v3/role_assignments?group.id=9394c6da936942d7814fb34ce09df006&scope.project.id=04a8a1a229ca4dd7aa955d90ed2e86c3&role.id=8e9576d16f194938b86a1d1e087895bd => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:26.630375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13c8d6-8d90-4b9c-b8fb-92b0531b71ed None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:26.640328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771093b1-84e5-46e4-a770-dc9858db879e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:26.641267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771093b1-84e5-46e4-a770-dc9858db879e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:26.641711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771093b1-84e5-46e4-a770-dc9858db879e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:26.642610 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771093b1-84e5-46e4-a770-dc9858db879e None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:26.643250 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771093b1-84e5-46e4-a770-dc9858db879e None admin] RBAC: Adding query filter params (name=member) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:26.643558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771093b1-84e5-46e4-a770-dc9858db879e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:26.644996 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771093b1-84e5-46e4-a770-dc9858db879e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:26.660526 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 218/437] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:09:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:26.661565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771093b1-84e5-46e4-a770-dc9858db879e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:26.733644 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:26.745344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:26.789891 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13c8d6-8d90-4b9c-b8fb-92b0531b71ed None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:26.791376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771093b1-84e5-46e4-a770-dc9858db879e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:26.833082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d2311-bf59-422d-880a-da99d2c4a391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:26.834011 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d2311-bf59-422d-880a-da99d2c4a391 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:26.834011 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d2311-bf59-422d-880a-da99d2c4a391 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:26.834258 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d2311-bf59-422d-880a-da99d2c4a391 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:26.846298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5faf7-d50d-4927-bf38-a95a1bd313c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:26.846974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5faf7-d50d-4927-bf38-a95a1bd313c5 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:26.847255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5faf7-d50d-4927-bf38-a95a1bd313c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:26.847526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5faf7-d50d-4927-bf38-a95a1bd313c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:26.861782 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5faf7-d50d-4927-bf38-a95a1bd313c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:26.866923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-d3a5faf7-d50d-4927-bf38-a95a1bd313c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:09:26.866923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:26.866923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:26.866923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:26.866923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:26.866923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:26.866923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:26.866923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:26.866923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:26.866923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:26.866923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:26.866923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:26.866923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:26.866923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:26.866923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:26.866923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:26.867876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:26.867876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:26.867876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:26.867876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:26.867876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:26.867876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:26.867876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:26.867876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:26.867876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:26.867876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:26.867876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:26.867876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:26.867876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:26.867876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:26.867876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:26.867876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:26.868663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:26.868663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:26.868663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:26.868663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:26.868663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:26.868663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:26.868663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:26.868663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:26.868663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:26.868663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:26.868663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:09:26.868663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:26.868663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 220/438] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:09:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:26.877115 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5faf7-d50d-4927-bf38-a95a1bd313c5 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:26.942760 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d2311-bf59-422d-880a-da99d2c4a391 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:26.944249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d2311-bf59-422d-880a-da99d2c4a391 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:27.020756 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5faf7-d50d-4927-bf38-a95a1bd313c5 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:27.030443 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:27.059149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:27.067084 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a3334-50bd-413c-a401-6a3f18b85680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:27.067705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a3334-50bd-413c-a401-6a3f18b85680 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:27.067987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a3334-50bd-413c-a401-6a3f18b85680 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:27.068262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a3334-50bd-413c-a401-6a3f18b85680 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:27.068933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a3334-50bd-413c-a401-6a3f18b85680 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:27.069292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a3334-50bd-413c-a401-6a3f18b85680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:27.070706 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a3334-50bd-413c-a401-6a3f18b85680 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:27.080318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 221/439] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:27.081645 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8456c-7352-4390-9de9-7a8bbeda33a1 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:27.081938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8456c-7352-4390-9de9-7a8bbeda33a1 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:27.082212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b8456c-7352-4390-9de9-7a8bbeda33a1 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:27.083595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 222/440] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:27.086911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8456c-7352-4390-9de9-7a8bbeda33a1 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:27.092823 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 219/441] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:27.094171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bed4b-9342-4801-a9cf-432c1a2a8358 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:27.094461 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bed4b-9342-4801-a9cf-432c1a2a8358 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:27.094730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bed4b-9342-4801-a9cf-432c1a2a8358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:27.116485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-fa3bed4b-9342-4801-a9cf-432c1a2a8358 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:27.117731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-fa3bed4b-9342-4801-a9cf-432c1a2a8358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:27.192605 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 220/442] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:27.193556 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bed4b-9342-4801-a9cf-432c1a2a8358 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:27.213332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b8456c-7352-4390-9de9-7a8bbeda33a1 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:27.253330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6360ef0-e678-466c-9de0-bfd8cf14fbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:27.253943 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6360ef0-e678-466c-9de0-bfd8cf14fbc5 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:27.254672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6360ef0-e678-466c-9de0-bfd8cf14fbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:27.254950 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6360ef0-e678-466c-9de0-bfd8cf14fbc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:27.303219 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:27.307909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bed4b-9342-4801-a9cf-432c1a2a8358 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:27.333507 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:27.347615 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416dc5b-3fcf-4e9c-a25d-1e38f38fb587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:27.347907 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6360ef0-e678-466c-9de0-bfd8cf14fbc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:27.348182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416dc5b-3fcf-4e9c-a25d-1e38f38fb587 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:27.348466 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416dc5b-3fcf-4e9c-a25d-1e38f38fb587 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:27.348747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416dc5b-3fcf-4e9c-a25d-1e38f38fb587 None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:27.349919 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416dc5b-3fcf-4e9c-a25d-1e38f38fb587 None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:27.350087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6360ef0-e678-466c-9de0-bfd8cf14fbc5 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:27.351346 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416dc5b-3fcf-4e9c-a25d-1e38f38fb587 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:27.352983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 221/443] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:27 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:27.358998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416dc5b-3fcf-4e9c-a25d-1e38f38fb587 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:27.464937 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416dc5b-3fcf-4e9c-a25d-1e38f38fb587 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:27.479525 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 223/444] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 12:09:27.480671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6360ef0-e678-466c-9de0-bfd8cf14fbc5 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:27.499118 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387c35a-1768-4bbb-815e-20c7ea73dbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:27.499746 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387c35a-1768-4bbb-815e-20c7ea73dbda None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:27.500027 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387c35a-1768-4bbb-815e-20c7ea73dbda None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:27.500292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387c35a-1768-4bbb-815e-20c7ea73dbda None admin] PATH_INFO: `/v3/groups/9394c6da936942d7814fb34ce09df006` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:27.502170 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5387c35a-1768-4bbb-815e-20c7ea73dbda None admin] RBAC: Authorizing `identity:get_group(group_id=9394c6da936942d7814fb34ce09df006)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:27.503743 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5387c35a-1768-4bbb-815e-20c7ea73dbda None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:27.505326 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 222/445] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 12:09:27 2022] GET /identity/v3/groups/9394c6da936942d7814fb34ce09df006 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:27.506082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387c35a-1768-4bbb-815e-20c7ea73dbda None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:27.539581 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:27.575256 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:27.585830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6360ef0-e678-466c-9de0-bfd8cf14fbc5 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:27.605849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387c35a-1768-4bbb-815e-20c7ea73dbda None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:27.622024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b21a9-46f4-4669-98fe-9c63bd1be463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:27.622659 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b21a9-46f4-4669-98fe-9c63bd1be463 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:27.623049 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b21a9-46f4-4669-98fe-9c63bd1be463 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:27.623510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b21a9-46f4-4669-98fe-9c63bd1be463 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:27.634325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b21a9-46f4-4669-98fe-9c63bd1be463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:27.639107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-043b21a9-46f4-4669-98fe-9c63bd1be463 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:09:27.639107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:27.639107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:27.639107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:27.639107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:27.639107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:27.639107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:27.639107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:27.639107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:27.639107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:27.639107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:27.639107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:27.639107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:27.639107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:27.639107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:27.639107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:27.640181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:27.640181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:27.640181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:27.640181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:27.640181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:27.640181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:27.640181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:27.640181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:27.640181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:27.640181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:27.640181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:27.640181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:27.640181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:27.640181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:27.640181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:27.640181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:27.641005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:27.641005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:27.641005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:27.641005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:27.641005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:27.641005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:27.641005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:27.641005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:27.641005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:27.641005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:27.641005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:09:27.641005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:27.641005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 224/446] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:09:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:27.641708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b21a9-46f4-4669-98fe-9c63bd1be463 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:27.647217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60432d-7f02-4d98-a48b-f31004053d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:27.647865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60432d-7f02-4d98-a48b-f31004053d21 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:27.648152 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60432d-7f02-4d98-a48b-f31004053d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:27.648431 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60432d-7f02-4d98-a48b-f31004053d21 None admin] PATH_INFO: `/v3/projects/81a7733885564252af45b01ea11bfed2` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:27.649715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60432d-7f02-4d98-a48b-f31004053d21 None admin] RBAC: Authorizing `identity:get_project(project_id=81a7733885564252af45b01ea11bfed2)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:27.651182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60432d-7f02-4d98-a48b-f31004053d21 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:27.652814 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 223/447] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:27 2022] GET /identity/v3/projects/81a7733885564252af45b01ea11bfed2 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:27.653608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60432d-7f02-4d98-a48b-f31004053d21 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:27.743091 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b21a9-46f4-4669-98fe-9c63bd1be463 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:27.753858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60432d-7f02-4d98-a48b-f31004053d21 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:27.755358 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:27.776822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74844a-ca8d-4b43-b93b-8a718266a20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:27.777470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74844a-ca8d-4b43-b93b-8a718266a20d None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:27.777746 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74844a-ca8d-4b43-b93b-8a718266a20d None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:27.778024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74844a-ca8d-4b43-b93b-8a718266a20d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:27.778731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74844a-ca8d-4b43-b93b-8a718266a20d None admin] RBAC: Adding query filter params (group.id=9394c6da936942d7814fb34ce09df006, role.id=2201e005d5ef449896a0e68a940678ec, scope.project.id=81a7733885564252af45b01ea11bfed2) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:27.779034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74844a-ca8d-4b43-b93b-8a718266a20d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:27.780449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74844a-ca8d-4b43-b93b-8a718266a20d None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:27.786805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 225/448] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 12:09:27 2022] GET /identity/v3/role_assignments?group.id=9394c6da936942d7814fb34ce09df006&scope.project.id=81a7733885564252af45b01ea11bfed2&role.id=2201e005d5ef449896a0e68a940678ec => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:27.788388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b55c96-8b17-49fa-b8d3-fda863c7bb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:27.789002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b55c96-8b17-49fa-b8d3-fda863c7bb64 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:27.789323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b55c96-8b17-49fa-b8d3-fda863c7bb64 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:27.789601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b55c96-8b17-49fa-b8d3-fda863c7bb64 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:27.790029 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74844a-ca8d-4b43-b93b-8a718266a20d None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:27.790337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b55c96-8b17-49fa-b8d3-fda863c7bb64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:27.790650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b55c96-8b17-49fa-b8d3-fda863c7bb64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:27.792058 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b55c96-8b17-49fa-b8d3-fda863c7bb64 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:27.804165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 224/449] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:27.809565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:27.811763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b55c96-8b17-49fa-b8d3-fda863c7bb64 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:27.906834 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74844a-ca8d-4b43-b93b-8a718266a20d None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:27.928196 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b55c96-8b17-49fa-b8d3-fda863c7bb64 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:27.944735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b110bf-85d8-4da0-bb14-d93b85998abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:27.945597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b110bf-85d8-4da0-bb14-d93b85998abf None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:27.945876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b110bf-85d8-4da0-bb14-d93b85998abf None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:27.946575 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b110bf-85d8-4da0-bb14-d93b85998abf None admin] PATH_INFO: `/v3/groups/9394c6da936942d7814fb34ce09df006` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:27.947919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b110bf-85d8-4da0-bb14-d93b85998abf None admin] RBAC: Authorizing `identity:get_group(group_id=9394c6da936942d7814fb34ce09df006)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:27.949427 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b110bf-85d8-4da0-bb14-d93b85998abf None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:27.955867 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 226/450] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 12:09:27 2022] GET /identity/v3/groups/9394c6da936942d7814fb34ce09df006 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:27.958100 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b110bf-85d8-4da0-bb14-d93b85998abf None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:27.970723 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b785d-b532-4719-a5b3-aca7d9515e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:27.971575 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b785d-b532-4719-a5b3-aca7d9515e23 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:27.971976 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b785d-b532-4719-a5b3-aca7d9515e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:27.972254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b785d-b532-4719-a5b3-aca7d9515e23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:28.014216 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:28.057031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b785d-b532-4719-a5b3-aca7d9515e23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:28.058521 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b785d-b532-4719-a5b3-aca7d9515e23 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:28.062257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:28.085865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b110bf-85d8-4da0-bb14-d93b85998abf None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:28.131639 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c810ad-fe84-4f08-9214-cba6597d2773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:28.135986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c810ad-fe84-4f08-9214-cba6597d2773 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:28.135986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c810ad-fe84-4f08-9214-cba6597d2773 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:28.135986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c810ad-fe84-4f08-9214-cba6597d2773 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:28.135986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c810ad-fe84-4f08-9214-cba6597d2773 None admin] RBAC: Adding query filter params (group.id=9394c6da936942d7814fb34ce09df006, role.id=8e9576d16f194938b86a1d1e087895bd, scope.project.id=81a7733885564252af45b01ea11bfed2) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:28.135986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c810ad-fe84-4f08-9214-cba6597d2773 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:28.136355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c810ad-fe84-4f08-9214-cba6597d2773 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:28.143783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 227/451] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 12:09:27 2022] GET /identity/v3/role_assignments?group.id=9394c6da936942d7814fb34ce09df006&scope.project.id=81a7733885564252af45b01ea11bfed2&role.id=8e9576d16f194938b86a1d1e087895bd => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:28.181031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 225/452] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 12:09:28.186610 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c810ad-fe84-4f08-9214-cba6597d2773 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:28.293712 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c810ad-fe84-4f08-9214-cba6597d2773 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:28.331561 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606598eb-3a07-4b50-823b-d9e69fa80110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:28.332196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606598eb-3a07-4b50-823b-d9e69fa80110 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:28.332471 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606598eb-3a07-4b50-823b-d9e69fa80110 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:28.332743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606598eb-3a07-4b50-823b-d9e69fa80110 None admin] PATH_INFO: `/v3/projects/2fc00be567cf4966a588e131d48e5a69` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:28.333871 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606598eb-3a07-4b50-823b-d9e69fa80110 None admin] RBAC: Authorizing `identity:get_project(project_id=2fc00be567cf4966a588e131d48e5a69)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:28.335280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606598eb-3a07-4b50-823b-d9e69fa80110 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:28.336864 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 228/453] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:09:28 2022] GET /identity/v3/projects/2fc00be567cf4966a588e131d48e5a69 => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:28.342415 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b785d-b532-4719-a5b3-aca7d9515e23 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:28.405097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:28.444837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b785d-b532-4719-a5b3-aca7d9515e23 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:28.480292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbab069-f228-4525-a92c-c9d38f7212e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:28.480934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbab069-f228-4525-a92c-c9d38f7212e2 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:28.481267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbab069-f228-4525-a92c-c9d38f7212e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:28.481531 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbab069-f228-4525-a92c-c9d38f7212e2 None admin] PATH_INFO: `/v3/groups/24e4eeee922c4b9b8d6db02bd993416f` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:28.482639 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbab069-f228-4525-a92c-c9d38f7212e2 None admin] RBAC: Authorizing `identity:get_group(group_id=24e4eeee922c4b9b8d6db02bd993416f)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:28.484095 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbab069-f228-4525-a92c-c9d38f7212e2 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:28.485756 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 226/454] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Sep 28 12:09:28 2022] GET /identity/v3/groups/24e4eeee922c4b9b8d6db02bd993416f => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:28.492010 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606598eb-3a07-4b50-823b-d9e69fa80110 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:28.510257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:28.592074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606598eb-3a07-4b50-823b-d9e69fa80110 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:28.623821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78000bc-76bf-4c31-8e5e-4eb3fe598f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:28.624416 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78000bc-76bf-4c31-8e5e-4eb3fe598f73 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:28.624696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78000bc-76bf-4c31-8e5e-4eb3fe598f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:28.624965 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78000bc-76bf-4c31-8e5e-4eb3fe598f73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:28.625708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78000bc-76bf-4c31-8e5e-4eb3fe598f73 None admin] RBAC: Adding query filter params (group.id=24e4eeee922c4b9b8d6db02bd993416f, role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=2fc00be567cf4966a588e131d48e5a69) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:28.626014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78000bc-76bf-4c31-8e5e-4eb3fe598f73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:28.627947 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78000bc-76bf-4c31-8e5e-4eb3fe598f73 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:28.633503 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 229/455] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Sep 28 12:09:28 2022] GET /identity/v3/role_assignments?group.id=24e4eeee922c4b9b8d6db02bd993416f&scope.project.id=2fc00be567cf4966a588e131d48e5a69&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:31.185295 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43227cc-e3b9-4842-8777-9fd9b71bdaaa None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:31.185816 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43227cc-e3b9-4842-8777-9fd9b71bdaaa None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:31.185816 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43227cc-e3b9-4842-8777-9fd9b71bdaaa None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:31.187243 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 227/456] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:31.203118 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3802906-0f6a-48f4-a087-9b8b87120453 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:31.203634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3802906-0f6a-48f4-a087-9b8b87120453 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:31.203634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3802906-0f6a-48f4-a087-9b8b87120453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:31.226121 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-e3802906-0f6a-48f4-a087-9b8b87120453 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:31.227307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e3802906-0f6a-48f4-a087-9b8b87120453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:31.319260 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 230/457] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:31.565482 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023da521-4b53-45b2-b026-47c1bfcfb66c None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:31.565896 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023da521-4b53-45b2-b026-47c1bfcfb66c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:31.566076 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023da521-4b53-45b2-b026-47c1bfcfb66c None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:31.567658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 228/458] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:31.572945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51690504-cf5b-4f08-83de-c2ecf6ab3411 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:31.573247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51690504-cf5b-4f08-83de-c2ecf6ab3411 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:31.573518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51690504-cf5b-4f08-83de-c2ecf6ab3411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:31.601369 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-51690504-cf5b-4f08-83de-c2ecf6ab3411 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:31.602546 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-51690504-cf5b-4f08-83de-c2ecf6ab3411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:31.604586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:31.666018 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a2da0-15ff-49dd-839d-1da8c5e60153 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:31.666290 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a2da0-15ff-49dd-839d-1da8c5e60153 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:31.666569 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a2da0-15ff-49dd-839d-1da8c5e60153 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:31.668178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 229/459] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:31.682543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058f0a3-d94a-4721-b802-4ce2c731f7f4 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:31.682834 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058f0a3-d94a-4721-b802-4ce2c731f7f4 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:31.683114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058f0a3-d94a-4721-b802-4ce2c731f7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:31.699330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 231/460] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:31.712935 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-7058f0a3-d94a-4721-b802-4ce2c731f7f4 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:31.714152 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7058f0a3-d94a-4721-b802-4ce2c731f7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:31.737009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51690504-cf5b-4f08-83de-c2ecf6ab3411 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:31.833731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 230/461] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:31.847344 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51690504-cf5b-4f08-83de-c2ecf6ab3411 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:31.883222 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a286a5-1e5f-4d32-96a2-fd328948ef1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:31.883859 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a286a5-1e5f-4d32-96a2-fd328948ef1e None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:31.884204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a286a5-1e5f-4d32-96a2-fd328948ef1e None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:31.884479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a286a5-1e5f-4d32-96a2-fd328948ef1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:31.891528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:31.895998 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a286a5-1e5f-4d32-96a2-fd328948ef1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:31.897473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a286a5-1e5f-4d32-96a2-fd328948ef1e None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:31.912761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-64a286a5-1e5f-4d32-96a2-fd328948ef1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:31.912761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:31.912761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:31.912761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:31.912761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:31.912761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:31.912761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:31.912761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:31.912761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:31.912761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:31.912761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:31.912761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:31.912761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:31.912761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:31.912761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:31.912761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:31.913720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:31.913720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:31.913720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:31.913720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:31.913720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:31.913720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:31.913720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:31.913720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:31.913720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:31.913720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:31.913720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:31.913720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:31.913720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:31.913720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:31.913720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:31.913720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:31.913720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:31.914583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:31.914583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:31.914583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:31.914583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 232/462] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:31.919800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058f0a3-d94a-4721-b802-4ce2c731f7f4 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:32.018205 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:32.071794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058f0a3-d94a-4721-b802-4ce2c731f7f4 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:32.077942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a286a5-1e5f-4d32-96a2-fd328948ef1e None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:32.133938 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b2d9a-0e1a-4299-a8e4-dd90deebc716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:32.134478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b2d9a-0e1a-4299-a8e4-dd90deebc716 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:32.134762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b2d9a-0e1a-4299-a8e4-dd90deebc716 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:32.135039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b2d9a-0e1a-4299-a8e4-dd90deebc716 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:32.135693 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b2d9a-0e1a-4299-a8e4-dd90deebc716 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:32.135999 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b2d9a-0e1a-4299-a8e4-dd90deebc716 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:32.137415 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b2d9a-0e1a-4299-a8e4-dd90deebc716 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:32.173183 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 231/463] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:32.184138 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b2d9a-0e1a-4299-a8e4-dd90deebc716 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:32.194158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a286a5-1e5f-4d32-96a2-fd328948ef1e None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:32.231184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead7d62-b0f6-4a44-9b61-8750f254394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:32.231833 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead7d62-b0f6-4a44-9b61-8750f254394f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:32.232088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead7d62-b0f6-4a44-9b61-8750f254394f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:32.232366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead7d62-b0f6-4a44-9b61-8750f254394f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:32.244177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead7d62-b0f6-4a44-9b61-8750f254394f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:32.245958 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead7d62-b0f6-4a44-9b61-8750f254394f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:32.262796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-aead7d62-b0f6-4a44-9b61-8750f254394f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:32.262796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:32.262796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:32.262796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:32.262796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:32.262796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:32.262796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:32.262796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:32.262796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:32.262796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:32.262796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:32.262796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:32.262796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:32.262796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:32.262796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:32.262796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:32.263953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:32.263953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:32.263953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:32.263953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:32.263953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:32.263953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:32.263953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:32.263953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:32.263953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:32.263953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:32.263953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:32.263953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:32.263953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:32.263953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:32.263953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:32.263953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:32.263953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:32.265006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:32.265006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:32.265006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:32.267302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 233/464] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:32.273519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead7d62-b0f6-4a44-9b61-8750f254394f None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:32.326184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:32.354637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b2d9a-0e1a-4299-a8e4-dd90deebc716 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:32.381118 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:32.394191 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead7d62-b0f6-4a44-9b61-8750f254394f None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:32.395753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e78ce0-d8c0-4fed-a766-2e0529127642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:32.396373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e78ce0-d8c0-4fed-a766-2e0529127642 None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:32.396650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e78ce0-d8c0-4fed-a766-2e0529127642 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:32.396928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e78ce0-d8c0-4fed-a766-2e0529127642 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:32.397618 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e78ce0-d8c0-4fed-a766-2e0529127642 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:32.399062 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e78ce0-d8c0-4fed-a766-2e0529127642 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:32.424228 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 232/465] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:09:32 2022] POST /identity/v3/users => generated 246 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:09:32.426021 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ffd7c-e97b-432c-9633-d92256482614 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:32.426489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ffd7c-e97b-432c-9633-d92256482614 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:32.426938 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ffd7c-e97b-432c-9633-d92256482614 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:32.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 233/466] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:32.429424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ffd7c-e97b-432c-9633-d92256482614 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:32.439144 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd85ecf-bba5-4787-b2fd-2109ac453469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:32.439873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd85ecf-bba5-4787-b2fd-2109ac453469 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:32.440232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd85ecf-bba5-4787-b2fd-2109ac453469 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:32.440584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd85ecf-bba5-4787-b2fd-2109ac453469 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:32.441449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd85ecf-bba5-4787-b2fd-2109ac453469 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:32.441831 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd85ecf-bba5-4787-b2fd-2109ac453469 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:32.443411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd85ecf-bba5-4787-b2fd-2109ac453469 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:32.458642 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 234/467] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:32.461815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3475b-803a-4ed9-878d-0e1d78945c54 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:32.463204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3475b-803a-4ed9-878d-0e1d78945c54 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:32.465406 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3475b-803a-4ed9-878d-0e1d78945c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:32.492347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-2ec3475b-803a-4ed9-878d-0e1d78945c54 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:32.493828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2ec3475b-803a-4ed9-878d-0e1d78945c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:32.575586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ffd7c-e97b-432c-9633-d92256482614 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:32.588031 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 235/468] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:32.589647 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3475b-803a-4ed9-878d-0e1d78945c54 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:32.621685 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52517eed-a3bd-458b-8c03-7cfc0592fecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:32.622589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52517eed-a3bd-458b-8c03-7cfc0592fecf None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:32.623220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52517eed-a3bd-458b-8c03-7cfc0592fecf None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:32.623640 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52517eed-a3bd-458b-8c03-7cfc0592fecf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:32.625706 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:32.639654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52517eed-a3bd-458b-8c03-7cfc0592fecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:32.642476 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52517eed-a3bd-458b-8c03-7cfc0592fecf None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:32.659810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-52517eed-a3bd-458b-8c03-7cfc0592fecf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:32.659810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:32.659810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:32.659810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:32.659810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:32.659810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:32.659810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:32.659810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:32.659810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:32.659810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:32.659810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:32.659810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:32.659810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:32.659810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:32.659810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:32.659810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:32.660702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:32.660702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:32.660702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:32.660702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:32.660702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:32.660702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:32.660702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:32.660702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:32.660702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:32.660702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:32.660702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:32.660702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:32.660702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:32.660702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:32.660702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:32.660702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:32.660702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:32.662150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:32.662150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:32.662150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:32.662150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 234/469] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:32.662150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46ce09-c881-4637-8b88-e83b0178b175 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:32.662446 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46ce09-c881-4637-8b88-e83b0178b175 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:32.662651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46ce09-c881-4637-8b88-e83b0178b175 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:32.664096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 235/470] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:32.668463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46ce09-c881-4637-8b88-e83b0178b175 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:32.707019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:32.718729 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3475b-803a-4ed9-878d-0e1d78945c54 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:32.773478 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b878b-0ee4-4a42-b196-8c4a3bb8dabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:32.774089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b878b-0ee4-4a42-b196-8c4a3bb8dabe None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:32.774365 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b878b-0ee4-4a42-b196-8c4a3bb8dabe None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:32.774628 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b878b-0ee4-4a42-b196-8c4a3bb8dabe None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:32.775257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b878b-0ee4-4a42-b196-8c4a3bb8dabe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:32.776686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b878b-0ee4-4a42-b196-8c4a3bb8dabe None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:32.800280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46ce09-c881-4637-8b88-e83b0178b175 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:32.801631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 236/471] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:09:32 2022] POST /identity/v3/users => generated 248 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:09:32.803022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be609ff-d718-49cd-ae86-33b5256c135b None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:32.803307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be609ff-d718-49cd-ae86-33b5256c135b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:32.803576 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be609ff-d718-49cd-ae86-33b5256c135b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:32.825682 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-0be609ff-d718-49cd-ae86-33b5256c135b None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:32.826900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-0be609ff-d718-49cd-ae86-33b5256c135b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:32.842875 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e349eff-e93b-44e7-84c4-0962134b27a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:32.843881 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e349eff-e93b-44e7-84c4-0962134b27a4 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:32.844464 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e349eff-e93b-44e7-84c4-0962134b27a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:32.844977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e349eff-e93b-44e7-84c4-0962134b27a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:32.845988 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e349eff-e93b-44e7-84c4-0962134b27a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:32.846539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e349eff-e93b-44e7-84c4-0962134b27a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:32.848266 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e349eff-e93b-44e7-84c4-0962134b27a4 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:32.863486 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 236/472] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:32.868909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e349eff-e93b-44e7-84c4-0962134b27a4 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:32.915902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 237/473] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:32.965473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be609ff-d718-49cd-ae86-33b5256c135b None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:33.000504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:33.036092 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:33.049636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e349eff-e93b-44e7-84c4-0962134b27a4 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:33.092045 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a5536-848b-4855-82d8-2da0cb4db017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:33.093023 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a5536-848b-4855-82d8-2da0cb4db017 None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:33.093570 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a5536-848b-4855-82d8-2da0cb4db017 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:33.094035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a5536-848b-4855-82d8-2da0cb4db017 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:33.095389 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a5536-848b-4855-82d8-2da0cb4db017 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:33.097251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a5536-848b-4855-82d8-2da0cb4db017 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:33.128171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 237/474] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:09:32 2022] POST /identity/v3/users => generated 248 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:09:33.140884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be609ff-d718-49cd-ae86-33b5256c135b None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:33.195492 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee67094-f1d5-454d-aaa3-b15ba3ad5fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:33.196148 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee67094-f1d5-454d-aaa3-b15ba3ad5fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:33.196427 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee67094-f1d5-454d-aaa3-b15ba3ad5fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:33.196750 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee67094-f1d5-454d-aaa3-b15ba3ad5fc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:33.208939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee67094-f1d5-454d-aaa3-b15ba3ad5fc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:33.210541 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee67094-f1d5-454d-aaa3-b15ba3ad5fc5 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:33.228332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-fee67094-f1d5-454d-aaa3-b15ba3ad5fc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:33.228332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:33.228332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:33.228332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:33.228332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:33.228332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:33.228332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:33.228332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:33.228332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:33.228332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:33.228332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:33.228332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:33.228332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:33.228332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:33.228332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:33.228332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:33.229329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:33.229329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:33.229329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:33.229329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:33.229329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:33.229329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:33.229329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:33.229329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:33.229329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:33.229329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:33.229329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:33.229329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:33.229329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:33.229329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:33.229329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:33.229329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:33.229329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:33.230216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:33.230216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:33.230216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:33.233177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 238/475] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:33.236620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a5536-848b-4855-82d8-2da0cb4db017 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:33.251642 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee67094-f1d5-454d-aaa3-b15ba3ad5fc5 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:33.365161 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:33.404180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee67094-f1d5-454d-aaa3-b15ba3ad5fc5 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:33.405367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a5536-848b-4855-82d8-2da0cb4db017 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:33.447190 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c391c-7d58-44ae-96eb-fa0d22989ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:33.448041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c391c-7d58-44ae-96eb-fa0d22989ae0 None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:33.448488 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c391c-7d58-44ae-96eb-fa0d22989ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:33.448904 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c391c-7d58-44ae-96eb-fa0d22989ae0 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:33.449063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951a942-b358-4197-93c3-452ac95fe766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:33.449815 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951a942-b358-4197-93c3-452ac95fe766 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:33.450030 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c391c-7d58-44ae-96eb-fa0d22989ae0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:33.450269 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951a942-b358-4197-93c3-452ac95fe766 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:33.450693 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951a942-b358-4197-93c3-452ac95fe766 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:33.451445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951a942-b358-4197-93c3-452ac95fe766 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:33.451846 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951a942-b358-4197-93c3-452ac95fe766 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:33.452136 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c391c-7d58-44ae-96eb-fa0d22989ae0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:33.453411 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951a942-b358-4197-93c3-452ac95fe766 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:33.464156 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.sql.core [None req-5b8c391c-7d58-44ae-96eb-fa0d22989ae0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 28 12:09:33.464156 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 28 12:09:33.464156 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [parameters: {'id': '47e0afd0afda41dba870ecb18354322f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 28 12:09:33.464156 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81384) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 28 12:09:33.469305 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 238/476] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:33.475233 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951a942-b358-4197-93c3-452ac95fe766 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:33.484757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-5b8c391c-7d58-44ae-96eb-fa0d22989ae0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 28 12:09:33.484757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:33.484757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 12:09:33.484757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 12:09:33.484757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 12:09:33.484757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 12:09:33.484757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 12:09:33.484757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 12:09:33.484757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 12:09:33.484757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application conn.query(q) Sep 28 12:09:33.484757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 12:09:33.484757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 12:09:33.484757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 12:09:33.484757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application result.read() Sep 28 12:09:33.484757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return ref.to_dict() Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 12:09:33.486307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application next(self.gen) Sep 28 12:09:33.487411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 28 12:09:33.487411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application yield resource Sep 28 12:09:33.487411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 12:09:33.487411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application next(self.gen) Sep 28 12:09:33.487411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 28 12:09:33.487411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self.session.rollback() Sep 28 12:09:33.487411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 28 12:09:33.487411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self.force_reraise() Sep 28 12:09:33.487411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 28 12:09:33.487411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise self.value Sep 28 12:09:33.487411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 28 12:09:33.487411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 28 12:09:33.487411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 28 12:09:33.487411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application session.commit() Sep 28 12:09:33.487411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 28 12:09:33.487411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 28 12:09:33.487411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 28 12:09:33.488476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._prepare_impl() Sep 28 12:09:33.488476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 28 12:09:33.488476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self.session.flush() Sep 28 12:09:33.488476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 28 12:09:33.488476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._flush(objects) Sep 28 12:09:33.488476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 28 12:09:33.488476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 28 12:09:33.488476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 28 12:09:33.488476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application compat.raise_( Sep 28 12:09:33.488476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 12:09:33.488476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception Sep 28 12:09:33.488476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 28 12:09:33.488476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application flush_context.execute() Sep 28 12:09:33.488476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 28 12:09:33.488476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rec.execute(self) Sep 28 12:09:33.488476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 28 12:09:33.488476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 28 12:09:33.489532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 28 12:09:33.489532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 28 12:09:33.489532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 28 12:09:33.489532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 28 12:09:33.489532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 28 12:09:33.489532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 28 12:09:33.489532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 28 12:09:33.489532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 28 12:09:33.489532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 28 12:09:33.489532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 28 12:09:33.489532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 28 12:09:33.489532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 28 12:09:33.489532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 28 12:09:33.489532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 28 12:09:33.489532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 12:09:33.489532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception Sep 28 12:09:33.490514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 12:09:33.490514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 12:09:33.490514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 12:09:33.490514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 12:09:33.490514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 12:09:33.490514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 12:09:33.490514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 12:09:33.490514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application conn.query(q) Sep 28 12:09:33.490514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 12:09:33.490514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 12:09:33.490514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 12:09:33.490514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application result.read() Sep 28 12:09:33.490514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 12:09:33.490514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 12:09:33.490514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 12:09:33.490514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 12:09:33.491473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 12:09:33.491473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 12:09:33.491473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 12:09:33.491473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 12:09:33.491473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 28 12:09:33.491473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 28 12:09:33.491473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [parameters: {'id': '47e0afd0afda41dba870ecb18354322f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 28 12:09:33.491473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 28 12:09:33.491473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:33.491473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:09:33.491473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:33.491473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:33.491473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:33.491473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:33.491473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:33.491473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:33.492421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:33.492421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:33.492421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:33.492421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:33.492421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:33.492421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:33.492421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 28 12:09:33.492421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 28 12:09:33.492421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:33.492421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:33.492421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 28 12:09:33.492421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 28 12:09:33.492421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 28 12:09:33.492421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 28 12:09:33.492421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 28 12:09:33.492421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:33.495867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:33.496907 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 239/477] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Sep 28 12:09:33 2022] POST /identity/v3/roles => generated 144 bytes in 246 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:33.505248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c391c-7d58-44ae-96eb-fa0d22989ae0 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:33.622648 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951a942-b358-4197-93c3-452ac95fe766 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:33.635708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c391c-7d58-44ae-96eb-fa0d22989ae0 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:33.672578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0670a1-9cbb-4602-bdd5-8cdddac8d9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:33.673492 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0670a1-9cbb-4602-bdd5-8cdddac8d9a8 None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:33.673909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0670a1-9cbb-4602-bdd5-8cdddac8d9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:33.674350 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0670a1-9cbb-4602-bdd5-8cdddac8d9a8 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:33.674460 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e91f6b-c7da-4e8b-a16a-1cf484d22fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:33.675235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0670a1-9cbb-4602-bdd5-8cdddac8d9a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:33.675343 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e91f6b-c7da-4e8b-a16a-1cf484d22fac None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:33.675724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e91f6b-c7da-4e8b-a16a-1cf484d22fac None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:33.676082 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e91f6b-c7da-4e8b-a16a-1cf484d22fac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:33.677211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0670a1-9cbb-4602-bdd5-8cdddac8d9a8 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:33.678381 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:33.691873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e91f6b-c7da-4e8b-a16a-1cf484d22fac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:33.698582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-f1e91f6b-c7da-4e8b-a16a-1cf484d22fac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:09:33.698582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:33.698582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:33.698582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:33.698582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:33.698582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:33.698582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:33.698582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:33.698582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:33.698582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:33.698582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:33.698582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:33.698582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:33.698582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:33.698582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:33.698582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:33.700043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:33.700043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:33.700043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:33.700043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:33.700043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:33.700043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:33.700043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:33.700043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:33.700043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:33.700043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:33.700043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:33.700043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:33.700043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:33.700043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:33.700043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:33.700043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:33.701038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:33.701038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:33.701038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:33.701038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:33.701038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:33.701038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:33.701038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:33.701038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:33.701038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:33.701038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:33.701038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 12:09:33.701038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:33.702736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 240/478] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:09:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:33.709062 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e91f6b-c7da-4e8b-a16a-1cf484d22fac None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:33.709300 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 239/479] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:09:33 2022] POST /identity/v3/users => generated 249 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:09:33.827153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e91f6b-c7da-4e8b-a16a-1cf484d22fac None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:33.864938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c53b1-f0bc-40bd-94e2-34092e9a8273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:33.865827 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c53b1-f0bc-40bd-94e2-34092e9a8273 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:33.866246 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c53b1-f0bc-40bd-94e2-34092e9a8273 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:33.866655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c53b1-f0bc-40bd-94e2-34092e9a8273 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:33.867472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c53b1-f0bc-40bd-94e2-34092e9a8273 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:33.867943 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c53b1-f0bc-40bd-94e2-34092e9a8273 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:33.869590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c53b1-f0bc-40bd-94e2-34092e9a8273 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:33.880419 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 241/480] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:34.957745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e34319-47e7-4b9d-b685-ffb0f1f2f798 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:34.958307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e34319-47e7-4b9d-b685-ffb0f1f2f798 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:34.958307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e34319-47e7-4b9d-b685-ffb0f1f2f798 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:34.959874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 240/481] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:34.965335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1bbb0-a122-45c0-be01-bc3206a8ade7 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:34.965761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1bbb0-a122-45c0-be01-bc3206a8ade7 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:34.965987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1bbb0-a122-45c0-be01-bc3206a8ade7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:34.993728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-5ac1bbb0-a122-45c0-be01-bc3206a8ade7 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:34.994964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5ac1bbb0-a122-45c0-be01-bc3206a8ade7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:35.059884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:35.086225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 242/482] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:35.490466 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e34319-47e7-4b9d-b685-ffb0f1f2f798 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:35.571209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:35.622952 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e34319-47e7-4b9d-b685-ffb0f1f2f798 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:35.653245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e11b0-e28d-4f6d-95ef-15b7523e922b None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:35.653796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e11b0-e28d-4f6d-95ef-15b7523e922b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:35.654344 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e11b0-e28d-4f6d-95ef-15b7523e922b None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:35.655639 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 243/483] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:35.663210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca8c1d-0110-4bdd-bd1e-0b853059f0c9 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:35.663490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca8c1d-0110-4bdd-bd1e-0b853059f0c9 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:35.663756 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca8c1d-0110-4bdd-bd1e-0b853059f0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:35.669115 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b838e-3f54-462a-8186-e7fad7007fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:35.669770 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b838e-3f54-462a-8186-e7fad7007fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:35.670083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b838e-3f54-462a-8186-e7fad7007fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:35.670309 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b838e-3f54-462a-8186-e7fad7007fd4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:35.684065 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b838e-3f54-462a-8186-e7fad7007fd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:35.690293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-420b838e-3f54-462a-8186-e7fad7007fd4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:35.690293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:35.690293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:35.690293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:35.690293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:35.690293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:35.690293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:35.690293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:35.690293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:35.690293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:35.690293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:35.690293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:35.690293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:35.690293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:35.690293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:35.690293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:35.691209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:35.691209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:35.691209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:35.691209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:35.691209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:35.691209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:35.691209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:35.691209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:35.691209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:35.691209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:35.691209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:35.691209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:35.691209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:35.691209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:35.691209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:35.691209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:35.692024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:35.692024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:35.692024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:35.692024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:35.692024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:35.692024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:35.692024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:35.692024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:35.692024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:35.692024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:35.692024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:35.692024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:35.695025 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 241/484] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:35.699909 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-18ca8c1d-0110-4bdd-bd1e-0b853059f0c9 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:35.699909 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-18ca8c1d-0110-4bdd-bd1e-0b853059f0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:35.701095 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b838e-3f54-462a-8186-e7fad7007fd4 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:35.807979 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 244/485] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:09:35.832492 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b838e-3f54-462a-8186-e7fad7007fd4 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:35.862999 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:35.880632 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d7b17-2b02-440a-ad50-f9bb59c9a4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:35.881603 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d7b17-2b02-440a-ad50-f9bb59c9a4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:35.882127 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d7b17-2b02-440a-ad50-f9bb59c9a4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:35.882662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974d7b17-2b02-440a-ad50-f9bb59c9a4a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:35.883534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d7b17-2b02-440a-ad50-f9bb59c9a4a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:35.884043 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d7b17-2b02-440a-ad50-f9bb59c9a4a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:35.885651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974d7b17-2b02-440a-ad50-f9bb59c9a4a8 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:35.900405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 242/486] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:09:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:35.906413 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca8c1d-0110-4bdd-bd1e-0b853059f0c9 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:36.034851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca8c1d-0110-4bdd-bd1e-0b853059f0c9 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:36.051699 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:36.070471 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d59ded-3c76-40b4-9d27-6d634d0bb4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:36.071027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d59ded-3c76-40b4-9d27-6d634d0bb4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:36.071299 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d59ded-3c76-40b4-9d27-6d634d0bb4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:36.071566 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d59ded-3c76-40b4-9d27-6d634d0bb4d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:36.082767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d59ded-3c76-40b4-9d27-6d634d0bb4d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:36.084267 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d59ded-3c76-40b4-9d27-6d634d0bb4d1 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:36.100226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-65d59ded-3c76-40b4-9d27-6d634d0bb4d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:36.100226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:36.100226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:36.100226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:36.100226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:36.100226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:36.100226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:36.100226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:36.100226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:36.100226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:36.100226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:36.100226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:36.100226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:36.100226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:36.100226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:36.100226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:36.101257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:36.101257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:36.101257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:36.101257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:36.101257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:36.101257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:36.101257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:36.101257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:36.101257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:36.101257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:36.101257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:36.101257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:36.101257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:36.101257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:36.101257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:36.101257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:36.101257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:36.102111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:36.102111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:36.102111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:36.102111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 245/487] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:36.108680 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d7b17-2b02-440a-ad50-f9bb59c9a4a8 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:36.166225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d59ded-3c76-40b4-9d27-6d634d0bb4d1 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:36.276522 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d59ded-3c76-40b4-9d27-6d634d0bb4d1 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:36.288244 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974d7b17-2b02-440a-ad50-f9bb59c9a4a8 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:36.311035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bbd17-2849-4577-a43a-f8275aec9a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:36.311657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bbd17-2849-4577-a43a-f8275aec9a96 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:36.311935 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bbd17-2849-4577-a43a-f8275aec9a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:36.312203 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bbd17-2849-4577-a43a-f8275aec9a96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:36.322772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bbd17-2849-4577-a43a-f8275aec9a96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:36.325735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c120746-25bd-419d-a491-34c243078626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:36.326356 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c120746-25bd-419d-a491-34c243078626 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:36.326646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c120746-25bd-419d-a491-34c243078626 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:36.326989 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c120746-25bd-419d-a491-34c243078626 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:36.327654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c120746-25bd-419d-a491-34c243078626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:36.327954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-839bbd17-2849-4577-a43a-f8275aec9a96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:36.327954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:36.327954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:36.327954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:36.327954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:36.327954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:36.327954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:36.327954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:36.327954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:36.327954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:36.327954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:36.327954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:36.327954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:36.327954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:36.327954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:36.327954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:36.328810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:36.328810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:36.328810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:36.328810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:36.328810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:36.328810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:36.328810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:36.328810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:36.328810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:36.328810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:36.328810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:36.328810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:36.328810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:36.328810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:36.328810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:36.328810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:36.330174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:36.330174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:36.330174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:36.330174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:36.330174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:36.330174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:36.330174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:36.330174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:36.330174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:36.330174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:36.330174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:36.330174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:36.330780 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c120746-25bd-419d-a491-34c243078626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:36.330847 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 246/488] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:36.330910 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c120746-25bd-419d-a491-34c243078626 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:36.331437 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826d390-16d9-4506-97e2-a86e17c75e98 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:36.331437 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826d390-16d9-4506-97e2-a86e17c75e98 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:36.331792 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826d390-16d9-4506-97e2-a86e17c75e98 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:36.333175 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 247/489] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:36.336922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826d390-16d9-4506-97e2-a86e17c75e98 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:36.348603 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 243/490] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:36.354053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1d982-ab7c-40cb-8f1e-92ef118ce2a2 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:36.354351 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1d982-ab7c-40cb-8f1e-92ef118ce2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:36.354629 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1d982-ab7c-40cb-8f1e-92ef118ce2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:36.367639 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:36.377616 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-fec1d982-ab7c-40cb-8f1e-92ef118ce2a2 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:36.379743 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-fec1d982-ab7c-40cb-8f1e-92ef118ce2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:36.433342 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:36.454780 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826d390-16d9-4506-97e2-a86e17c75e98 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:36.469042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 244/491] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:36.470821 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1d982-ab7c-40cb-8f1e-92ef118ce2a2 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:36.492621 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c6574-b23c-4a4a-abc3-26fe388d42c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:36.493677 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c6574-b23c-4a4a-abc3-26fe388d42c4 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:36.494177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c6574-b23c-4a4a-abc3-26fe388d42c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:36.494637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c6574-b23c-4a4a-abc3-26fe388d42c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:36.495511 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c6574-b23c-4a4a-abc3-26fe388d42c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:36.496004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c6574-b23c-4a4a-abc3-26fe388d42c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:36.497607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c6574-b23c-4a4a-abc3-26fe388d42c4 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:36.507349 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 248/492] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:09:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:36.513461 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c6574-b23c-4a4a-abc3-26fe388d42c4 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:36.645950 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1d982-ab7c-40cb-8f1e-92ef118ce2a2 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:36.688793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:36.695900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969de7df-6aff-4465-b665-4ae6018abc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:36.696532 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969de7df-6aff-4465-b665-4ae6018abc10 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:36.696808 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969de7df-6aff-4465-b665-4ae6018abc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:36.697082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969de7df-6aff-4465-b665-4ae6018abc10 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:36.700107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c6574-b23c-4a4a-abc3-26fe388d42c4 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:36.704869 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969de7df-6aff-4465-b665-4ae6018abc10 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:36.706514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969de7df-6aff-4465-b665-4ae6018abc10 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:36.719381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-969de7df-6aff-4465-b665-4ae6018abc10 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:09:36.719381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:36.719381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:36.719381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:36.719381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:36.719381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:36.719381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:36.719381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:36.719381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:36.719381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:36.719381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:36.719381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:36.719381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:09:36.719381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:09:36.719381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:09:36.719381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:36.720310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:36.721246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:36.721246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:36.721246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:09:36.721246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:09:36.721246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:09:36.721246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:09:36.721246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:09:36.721246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:09:36.721246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:09:36.721246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:36.721246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 245/493] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 12:09:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:36.722150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534ef5f-cf59-4ff5-9b6c-5d579c893354 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:36.722265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534ef5f-cf59-4ff5-9b6c-5d579c893354 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:36.722535 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534ef5f-cf59-4ff5-9b6c-5d579c893354 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:36.723805 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 246/494] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:36.724795 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054123fd-4498-41f1-aa23-27bbefdc750c None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:36.725071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054123fd-4498-41f1-aa23-27bbefdc750c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:36.725565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054123fd-4498-41f1-aa23-27bbefdc750c None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:36.729880 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 247/495] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:36.729880 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054123fd-4498-41f1-aa23-27bbefdc750c None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:36.746260 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8654bf-e447-4179-834e-4a03aaff6ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:36.747250 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8654bf-e447-4179-834e-4a03aaff6ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:36.747763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8654bf-e447-4179-834e-4a03aaff6ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:36.748255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8654bf-e447-4179-834e-4a03aaff6ad1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:36.762607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8654bf-e447-4179-834e-4a03aaff6ad1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:36.764261 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8654bf-e447-4179-834e-4a03aaff6ad1 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:36.765462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:36.780933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-1a8654bf-e447-4179-834e-4a03aaff6ad1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:36.780933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:36.780933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:36.780933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:36.780933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:36.780933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:36.780933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:36.780933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:36.780933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:36.780933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:36.780933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:36.780933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:36.780933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:36.780933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:36.780933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:36.780933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:36.781914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:36.781914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:36.781914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:36.781914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:36.781914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:36.781914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:36.781914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:36.781914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:36.781914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:36.781914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:36.781914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:36.781914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:36.781914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:36.781914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:36.781914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:36.781914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:36.781914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:36.782805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:36.782805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:36.782805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:36.782805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 249/496] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:36.785169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c91b9-3777-4582-9c35-514b1b7dffc3 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:36.785169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c91b9-3777-4582-9c35-514b1b7dffc3 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:36.785169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c91b9-3777-4582-9c35-514b1b7dffc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:36.802561 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-994c91b9-3777-4582-9c35-514b1b7dffc3 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:36.803744 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-994c91b9-3777-4582-9c35-514b1b7dffc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:36.832834 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054123fd-4498-41f1-aa23-27bbefdc750c None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:36.865814 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8dddbf-aa3d-4206-b233-23bd5e52b7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:36.866413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dddbf-aa3d-4206-b233-23bd5e52b7e5 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:36.866689 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dddbf-aa3d-4206-b233-23bd5e52b7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:36.866961 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dddbf-aa3d-4206-b233-23bd5e52b7e5 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:36.867678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8dddbf-aa3d-4206-b233-23bd5e52b7e5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:36.867981 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8dddbf-aa3d-4206-b233-23bd5e52b7e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:36.869186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 250/497] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:36.869437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8dddbf-aa3d-4206-b233-23bd5e52b7e5 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:36.870406 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19645487-2ba7-4ef0-8843-8825c90f205b None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:36.870684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19645487-2ba7-4ef0-8843-8825c90f205b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:36.870953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19645487-2ba7-4ef0-8843-8825c90f205b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:36.888727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-19645487-2ba7-4ef0-8843-8825c90f205b None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:36.889931 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-19645487-2ba7-4ef0-8843-8825c90f205b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:36.911285 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 248/498] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 12:09:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:36.911285 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8dddbf-aa3d-4206-b233-23bd5e52b7e5 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:36.973280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 251/499] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:36.974983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19645487-2ba7-4ef0-8843-8825c90f205b None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:36.978241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:37.005081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:37.015399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8dddbf-aa3d-4206-b233-23bd5e52b7e5 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:37.052450 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af4da4-5c74-446c-a5ac-bffa9dc3c40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:37.053062 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af4da4-5c74-446c-a5ac-bffa9dc3c40a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:37.053370 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af4da4-5c74-446c-a5ac-bffa9dc3c40a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:37.053641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af4da4-5c74-446c-a5ac-bffa9dc3c40a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:37.064621 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af4da4-5c74-446c-a5ac-bffa9dc3c40a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:37.069597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-40af4da4-5c74-446c-a5ac-bffa9dc3c40a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:37.069597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:37.069597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:37.069597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:37.069597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:37.069597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:37.069597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:37.069597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:37.069597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:37.069597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:37.069597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:37.069597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:37.069597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:37.069597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:37.069597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:37.069597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:37.070662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:37.070662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:37.070662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:37.070662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:37.070662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:37.070662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:37.070662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:37.070662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:37.070662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:37.070662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:37.070662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:37.070662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:37.070662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:37.070662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:37.070662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:37.070662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:37.076314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:37.076314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:37.076314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:37.076314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:37.076314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:37.076314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:37.076314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:37.076314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:37.076314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:37.076314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:37.076314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:37.076314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:37.076314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 249/500] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:37.076314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af4da4-5c74-446c-a5ac-bffa9dc3c40a None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:37.109062 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19645487-2ba7-4ef0-8843-8825c90f205b None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:37.145697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803f69b-54ad-493e-9a3b-8eba4645575a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:37.146299 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f69b-54ad-493e-9a3b-8eba4645575a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:37.146572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f69b-54ad-493e-9a3b-8eba4645575a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:37.146909 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803f69b-54ad-493e-9a3b-8eba4645575a None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:37.147533 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803f69b-54ad-493e-9a3b-8eba4645575a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:37.147830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803f69b-54ad-493e-9a3b-8eba4645575a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:37.149436 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803f69b-54ad-493e-9a3b-8eba4645575a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:37.164480 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 252/501] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:37.164480 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803f69b-54ad-493e-9a3b-8eba4645575a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:37.210230 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af4da4-5c74-446c-a5ac-bffa9dc3c40a None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:37.231730 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:37.246579 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e42ade-7e2d-49d0-bc47-fa0a0b22e15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:37.247179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e42ade-7e2d-49d0-bc47-fa0a0b22e15f None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:37.247456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e42ade-7e2d-49d0-bc47-fa0a0b22e15f None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:37.247725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e42ade-7e2d-49d0-bc47-fa0a0b22e15f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:37.269878 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:37.272373 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803f69b-54ad-493e-9a3b-8eba4645575a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:37.306199 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aeded5-e392-479d-95ff-aff960dd08b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:37.306799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aeded5-e392-479d-95ff-aff960dd08b1 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:37.307076 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aeded5-e392-479d-95ff-aff960dd08b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:37.307356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aeded5-e392-479d-95ff-aff960dd08b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:37.308004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aeded5-e392-479d-95ff-aff960dd08b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:37.308302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aeded5-e392-479d-95ff-aff960dd08b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:37.309715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aeded5-e392-479d-95ff-aff960dd08b1 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:37.316858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e42ade-7e2d-49d0-bc47-fa0a0b22e15f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:37.318017 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 253/502] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:09:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:37.322677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e42ade-7e2d-49d0-bc47-fa0a0b22e15f None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:37.322803 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aeded5-e392-479d-95ff-aff960dd08b1 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:37.417414 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 250/503] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:37.418167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e42ade-7e2d-49d0-bc47-fa0a0b22e15f None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:37.418595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aeded5-e392-479d-95ff-aff960dd08b1 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:37.441990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:37.452436 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1094aa-de45-42a8-802b-5668477039de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:37.453065 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1094aa-de45-42a8-802b-5668477039de None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:37.453388 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1094aa-de45-42a8-802b-5668477039de None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:37.453661 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1094aa-de45-42a8-802b-5668477039de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:37.464268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1094aa-de45-42a8-802b-5668477039de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:37.465731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1094aa-de45-42a8-802b-5668477039de None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:37.471123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:37.480662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-8d1094aa-de45-42a8-802b-5668477039de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:37.480662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:37.480662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:37.480662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:37.480662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:37.480662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:37.480662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:37.480662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:37.480662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:37.480662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:37.480662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:37.480662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:37.480662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:37.480662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:37.480662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:37.480662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:37.481595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:37.481595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:37.481595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:37.481595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:37.481595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:37.481595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:37.481595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:37.481595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:37.481595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:37.481595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:37.481595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:37.481595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:37.481595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:37.481595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:37.481595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:37.481595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:37.481595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:37.482482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:37.482482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:37.482482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:37.482482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 254/504] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:37.482705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1094aa-de45-42a8-802b-5668477039de None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:37.519093 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e42ade-7e2d-49d0-bc47-fa0a0b22e15f None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:37.552740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94284c49-ec2a-4031-a72c-e0d95cd7d815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:37.553401 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94284c49-ec2a-4031-a72c-e0d95cd7d815 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:37.553685 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94284c49-ec2a-4031-a72c-e0d95cd7d815 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:37.553968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94284c49-ec2a-4031-a72c-e0d95cd7d815 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:37.560863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94284c49-ec2a-4031-a72c-e0d95cd7d815 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:37.562368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94284c49-ec2a-4031-a72c-e0d95cd7d815 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:37.574054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-94284c49-ec2a-4031-a72c-e0d95cd7d815 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 12:09:37.574054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:37.574054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:37.574054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:37.574054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:37.574054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:37.574054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:37.574054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:37.574054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:37.574054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:37.574054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:37.574054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:37.574054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:09:37.574054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:09:37.574054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:09:37.574054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:37.574944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:37.575844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:37.575844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:37.575844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:09:37.575844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:09:37.575844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:09:37.575844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:09:37.575844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:09:37.575844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:09:37.575844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 12:09:37.575844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:37.575844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 251/505] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:37.576548 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94284c49-ec2a-4031-a72c-e0d95cd7d815 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:37.586437 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1094aa-de45-42a8-802b-5668477039de None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:37.619594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9c76d-73ff-4a6e-a81a-567373fac309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:37.620223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9c76d-73ff-4a6e-a81a-567373fac309 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:37.620501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9c76d-73ff-4a6e-a81a-567373fac309 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:37.620777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9c76d-73ff-4a6e-a81a-567373fac309 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:37.632024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9c76d-73ff-4a6e-a81a-567373fac309 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:37.637257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-24b9c76d-73ff-4a6e-a81a-567373fac309 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:37.637257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:37.637257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:37.637257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:37.637257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:37.637257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:37.637257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:37.637257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:37.637257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:37.637257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:37.637257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:37.637257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:37.637257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:37.637257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:37.637257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:37.637257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:37.638335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:37.638335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:37.638335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:37.638335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:37.638335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:37.638335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:37.638335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:37.638335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:37.638335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:37.638335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:37.638335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:37.638335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:37.638335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:37.638335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:37.638335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:37.638335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:37.639167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:37.639167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:37.639167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:37.639167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:37.639167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:37.639167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:37.639167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:37.639167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:37.639167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:37.639167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:37.639167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:37.639167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:37.639167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 255/506] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:37.639167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9c76d-73ff-4a6e-a81a-567373fac309 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:37.675795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:37.678436 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94284c49-ec2a-4031-a72c-e0d95cd7d815 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:37.691240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:37.712138 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9714063-2113-4de6-a963-9e85cea35b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:37.712757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9714063-2113-4de6-a963-9e85cea35b53 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:37.713036 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9714063-2113-4de6-a963-9e85cea35b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:37.713339 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9714063-2113-4de6-a963-9e85cea35b53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:37.724091 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9714063-2113-4de6-a963-9e85cea35b53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:37.725571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9714063-2113-4de6-a963-9e85cea35b53 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:37.740130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-e9714063-2113-4de6-a963-9e85cea35b53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:37.740130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:37.740130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:37.740130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:37.740130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:37.740130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:37.740130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:37.740130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:37.740130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:37.740130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:37.740130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:37.740130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:37.740130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:37.740130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:37.740130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:37.740130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:37.741217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:37.741217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:37.741217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:37.741217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:37.741217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:37.741217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:37.741217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:37.741217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:37.741217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:37.741217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:37.741217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:37.741217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:37.741217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:37.741217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:37.741217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:37.741217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:37.741217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:37.742079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:37.742079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:37.742079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:37.742242 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9c76d-73ff-4a6e-a81a-567373fac309 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:37.742328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 252/507] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:37.742328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9714063-2113-4de6-a963-9e85cea35b53 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:37.777575 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91e465-247f-45d5-8de9-06741998957f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:37.778191 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91e465-247f-45d5-8de9-06741998957f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:37.778465 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91e465-247f-45d5-8de9-06741998957f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:37.778730 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91e465-247f-45d5-8de9-06741998957f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:37.791127 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91e465-247f-45d5-8de9-06741998957f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:37.792672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91e465-247f-45d5-8de9-06741998957f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:37.807542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-fc91e465-247f-45d5-8de9-06741998957f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:37.807542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:37.807542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:37.807542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:37.807542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:37.807542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:37.807542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:37.807542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:37.807542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:37.807542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:37.807542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:37.807542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:37.807542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:37.807542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:37.807542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:37.807542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:37.808652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:37.808652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:37.808652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:37.808652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:37.808652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:37.808652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:37.808652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:37.808652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:37.808652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:37.808652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:37.808652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:37.808652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:37.808652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:37.808652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:37.808652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:37.808652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:37.808652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:37.809762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:37.809762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:37.809762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:37.813235 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 256/508] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:37.814244 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91e465-247f-45d5-8de9-06741998957f None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:37.839887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9714063-2113-4de6-a963-9e85cea35b53 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:37.872791 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7612ad1-021b-41e3-929a-41c801956790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:37.873424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7612ad1-021b-41e3-929a-41c801956790 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:37.873705 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7612ad1-021b-41e3-929a-41c801956790 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:37.873980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7612ad1-021b-41e3-929a-41c801956790 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:37.874650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7612ad1-021b-41e3-929a-41c801956790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:37.874954 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7612ad1-021b-41e3-929a-41c801956790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:37.876578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7612ad1-021b-41e3-929a-41c801956790 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:37.888039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 253/509] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:37.889163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7612ad1-021b-41e3-929a-41c801956790 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:37.906282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:37.914423 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:37.919390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91e465-247f-45d5-8de9-06741998957f None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:37.951718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd42e0-2ec0-4fbc-b0bd-4925450ef2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:37.952321 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd42e0-2ec0-4fbc-b0bd-4925450ef2a3 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:37.952596 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd42e0-2ec0-4fbc-b0bd-4925450ef2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:37.952865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd42e0-2ec0-4fbc-b0bd-4925450ef2a3 None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:37.953607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd42e0-2ec0-4fbc-b0bd-4925450ef2a3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:37.954463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd42e0-2ec0-4fbc-b0bd-4925450ef2a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:37.956071 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dd42e0-2ec0-4fbc-b0bd-4925450ef2a3 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:37.988938 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7612ad1-021b-41e3-929a-41c801956790 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:37.992545 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 257/510] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Sep 28 12:09:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:37.993440 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd42e0-2ec0-4fbc-b0bd-4925450ef2a3 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:38.024325 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf3a23-66c6-4c82-b262-511ae3c0bd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:38.024929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf3a23-66c6-4c82-b262-511ae3c0bd76 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:38.025234 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf3a23-66c6-4c82-b262-511ae3c0bd76 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:38.025508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf3a23-66c6-4c82-b262-511ae3c0bd76 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:38.026160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf3a23-66c6-4c82-b262-511ae3c0bd76 None admin] RBAC: Adding query filter params (name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:38.026464 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf3a23-66c6-4c82-b262-511ae3c0bd76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:38.027823 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf3a23-66c6-4c82-b262-511ae3c0bd76 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:38.036058 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 254/511] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:09:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:38.036829 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf3a23-66c6-4c82-b262-511ae3c0bd76 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:38.090681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd42e0-2ec0-4fbc-b0bd-4925450ef2a3 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:38.124196 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:38.125153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8c26f-c3d8-43af-a234-5b7493bcf371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:38.125801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8c26f-c3d8-43af-a234-5b7493bcf371 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:38.126043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8c26f-c3d8-43af-a234-5b7493bcf371 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:38.126349 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8c26f-c3d8-43af-a234-5b7493bcf371 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:38.126973 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8c26f-c3d8-43af-a234-5b7493bcf371 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:38.127271 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8c26f-c3d8-43af-a234-5b7493bcf371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:38.128648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8c26f-c3d8-43af-a234-5b7493bcf371 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:38.139211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf3a23-66c6-4c82-b262-511ae3c0bd76 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:38.139889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 258/512] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:38.140753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8c26f-c3d8-43af-a234-5b7493bcf371 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:38.150360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:38.180317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150fe0b6-1aed-46c7-a95b-4b051f6f325e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:38.180985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fe0b6-1aed-46c7-a95b-4b051f6f325e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:38.181303 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fe0b6-1aed-46c7-a95b-4b051f6f325e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:38.181580 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fe0b6-1aed-46c7-a95b-4b051f6f325e None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:38.182281 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150fe0b6-1aed-46c7-a95b-4b051f6f325e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:38.182590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150fe0b6-1aed-46c7-a95b-4b051f6f325e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:38.183969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150fe0b6-1aed-46c7-a95b-4b051f6f325e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:38.196187 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 255/513] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:38.196985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150fe0b6-1aed-46c7-a95b-4b051f6f325e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:38.247903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8c26f-c3d8-43af-a234-5b7493bcf371 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:38.281684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157d2f5-9266-464d-abb9-1a16b16f11b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:38.282405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157d2f5-9266-464d-abb9-1a16b16f11b3 None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:38.282684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157d2f5-9266-464d-abb9-1a16b16f11b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:38.282956 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157d2f5-9266-464d-abb9-1a16b16f11b3 None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:38.283585 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157d2f5-9266-464d-abb9-1a16b16f11b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:38.285004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9157d2f5-9266-464d-abb9-1a16b16f11b3 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:38.293639 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150fe0b6-1aed-46c7-a95b-4b051f6f325e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:38.306246 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 259/514] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:09:38 2022] POST /identity/v3/users => generated 247 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:09:38.307228 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157d2f5-9266-464d-abb9-1a16b16f11b3 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:38.332761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc609bcb-1f10-4549-bd72-8d71cb4a9a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:38.333529 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc609bcb-1f10-4549-bd72-8d71cb4a9a2c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:38.333874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc609bcb-1f10-4549-bd72-8d71cb4a9a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:38.334218 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc609bcb-1f10-4549-bd72-8d71cb4a9a2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:38.351688 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:38.380618 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:38.413625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc609bcb-1f10-4549-bd72-8d71cb4a9a2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:38.415973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc609bcb-1f10-4549-bd72-8d71cb4a9a2c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:38.429733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157d2f5-9266-464d-abb9-1a16b16f11b3 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:38.469324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee2263-cdc0-4baf-a580-c851e35ddeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:38.470214 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee2263-cdc0-4baf-a580-c851e35ddeeb None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:38.470582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee2263-cdc0-4baf-a580-c851e35ddeeb None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:38.471061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee2263-cdc0-4baf-a580-c851e35ddeeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:38.484230 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee2263-cdc0-4baf-a580-c851e35ddeeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:38.486010 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee2263-cdc0-4baf-a580-c851e35ddeeb None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:38.505495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-4fee2263-cdc0-4baf-a580-c851e35ddeeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:38.505495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:38.505495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:38.505495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:38.505495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:38.505495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:38.505495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:38.505495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:38.505495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:38.505495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:38.505495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:38.505495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:38.505495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:38.505495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:38.505495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:38.505495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:38.506604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:38.506604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:38.506604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:38.506604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:38.506604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:38.506604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:38.506604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:38.506604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:38.506604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:38.506604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:38.506604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:38.506604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:38.506604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:38.506604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:38.506604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:38.506604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:38.506604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:38.507449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:38.507449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:38.507449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:38.507449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 260/515] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:38.507449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee2263-cdc0-4baf-a580-c851e35ddeeb None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:38.539295 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 256/516] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:38.540142 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc609bcb-1f10-4549-bd72-8d71cb4a9a2c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:38.591364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:38.623461 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee2263-cdc0-4baf-a580-c851e35ddeeb None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:38.638549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:38.646731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc609bcb-1f10-4549-bd72-8d71cb4a9a2c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:38.661399 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46595dff-b1ae-48bf-9937-9f0b18eefbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:38.662028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46595dff-b1ae-48bf-9937-9f0b18eefbfa None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:38.662300 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46595dff-b1ae-48bf-9937-9f0b18eefbfa None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:38.662564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46595dff-b1ae-48bf-9937-9f0b18eefbfa None admin] PATH_INFO: `/v3/users/glance` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:38.670685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46595dff-b1ae-48bf-9937-9f0b18eefbfa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:38.672121 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46595dff-b1ae-48bf-9937-9f0b18eefbfa None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:38.681169 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611e3dc-9500-4aba-b0cf-306749aa77b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:38.681804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611e3dc-9500-4aba-b0cf-306749aa77b1 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:38.682081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611e3dc-9500-4aba-b0cf-306749aa77b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:38.682350 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611e3dc-9500-4aba-b0cf-306749aa77b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:38.685656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-46595dff-b1ae-48bf-9937-9f0b18eefbfa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 28 12:09:38.685656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:38.685656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:38.685656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:38.685656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:38.685656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:38.685656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:38.685656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:38.685656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:38.685656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:38.685656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:38.685656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:38.685656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:09:38.685656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:09:38.685656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:09:38.685656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:38.686554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:38.692509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:38.692509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:38.692509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:09:38.692509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:09:38.692509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:09:38.692509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:09:38.692509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:09:38.692509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:09:38.692509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 28 12:09:38.692509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:38.692509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 261/517] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:38.692509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46595dff-b1ae-48bf-9937-9f0b18eefbfa None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:38.693202 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611e3dc-9500-4aba-b0cf-306749aa77b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:38.693202 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611e3dc-9500-4aba-b0cf-306749aa77b1 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:38.704853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-b611e3dc-9500-4aba-b0cf-306749aa77b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:38.704853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:38.704853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:38.704853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:38.704853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:38.704853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:38.704853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:38.704853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:38.704853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:38.704853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:38.704853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:38.704853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:38.704853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:09:38.704853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:09:38.704853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:09:38.705837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:09:38.705837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:38.705837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:38.705837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:38.705837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:38.705837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:38.705837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:38.705837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:38.705837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:38.705837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:38.705837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:38.705837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:38.705837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:38.705837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:38.705837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:38.705837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:38.705837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:38.706739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:09:38.706739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:09:38.706739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:09:38.706739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:09:38.706739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:09:38.706739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:09:38.706739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:38.706739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:38.706739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 257/518] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:38.706739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611e3dc-9500-4aba-b0cf-306749aa77b1 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:38.804682 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46595dff-b1ae-48bf-9937-9f0b18eefbfa None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:38.818748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611e3dc-9500-4aba-b0cf-306749aa77b1 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:38.826618 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:38.845176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c801b0-1692-4d61-99da-1ac79e398e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:38.845911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c801b0-1692-4d61-99da-1ac79e398e61 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:38.846190 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c801b0-1692-4d61-99da-1ac79e398e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:38.846459 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c801b0-1692-4d61-99da-1ac79e398e61 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:38.847235 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c801b0-1692-4d61-99da-1ac79e398e61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:38.847558 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c801b0-1692-4d61-99da-1ac79e398e61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:38.849027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c801b0-1692-4d61-99da-1ac79e398e61 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:38.855603 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032d8fe-30c2-452f-b6c1-5166f737b829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:38.856737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032d8fe-30c2-452f-b6c1-5166f737b829 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:38.857006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032d8fe-30c2-452f-b6c1-5166f737b829 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:38.857308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032d8fe-30c2-452f-b6c1-5166f737b829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:38.865611 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 262/519] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:38.865611 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c801b0-1692-4d61-99da-1ac79e398e61 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:38.870112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032d8fe-30c2-452f-b6c1-5166f737b829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:38.871649 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032d8fe-30c2-452f-b6c1-5166f737b829 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:38.886995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-4032d8fe-30c2-452f-b6c1-5166f737b829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:38.886995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:38.886995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:38.886995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:38.886995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:38.886995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:38.886995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:38.886995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:38.886995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:38.886995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:38.886995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:38.886995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:38.886995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:38.886995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:38.886995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:38.886995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:38.887905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:38.887905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:38.887905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:38.887905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:38.887905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:38.887905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:38.887905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:38.887905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:38.887905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:38.887905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:38.887905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:38.887905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:38.887905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:38.887905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:38.887905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:38.887905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:38.887905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:38.888751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:38.888751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:38.888751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:38.888751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 258/520] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:38.888960 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032d8fe-30c2-452f-b6c1-5166f737b829 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:38.903269 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:38.975952 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c801b0-1692-4d61-99da-1ac79e398e61 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:38.994006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032d8fe-30c2-452f-b6c1-5166f737b829 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:39.012337 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e99513-2408-4688-86ef-da88504d3fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:39.013079 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e99513-2408-4688-86ef-da88504d3fab None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:39.013480 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e99513-2408-4688-86ef-da88504d3fab None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:39.013958 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e99513-2408-4688-86ef-da88504d3fab None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:39.014771 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e99513-2408-4688-86ef-da88504d3fab None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:39.015171 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e99513-2408-4688-86ef-da88504d3fab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:39.016846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e99513-2408-4688-86ef-da88504d3fab None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:39.027592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38469b-2a88-498d-b257-1577401084df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:39.028281 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38469b-2a88-498d-b257-1577401084df None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:39.028629 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38469b-2a88-498d-b257-1577401084df None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:39.028968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38469b-2a88-498d-b257-1577401084df None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:39.029761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38469b-2a88-498d-b257-1577401084df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:39.030131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38469b-2a88-498d-b257-1577401084df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:39.031624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38469b-2a88-498d-b257-1577401084df None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:39.040406 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 263/521] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Sep 28 12:09:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:39.040406 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e99513-2408-4688-86ef-da88504d3fab None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:39.055541 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 259/522] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:09:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:39.056208 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38469b-2a88-498d-b257-1577401084df None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:39.076576 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:39.146876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:39.157078 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e99513-2408-4688-86ef-da88504d3fab None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:39.166428 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38469b-2a88-498d-b257-1577401084df None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:39.198841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b14826-4b71-482b-a5c4-867009ec4a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:39.199596 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b14826-4b71-482b-a5c4-867009ec4a98 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:39.199945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b14826-4b71-482b-a5c4-867009ec4a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:39.200289 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b14826-4b71-482b-a5c4-867009ec4a98 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:39.202490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f9d7a-1cf7-4e81-9ea9-8d7b79d5fb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:39.203168 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f9d7a-1cf7-4e81-9ea9-8d7b79d5fb31 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:39.203508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f9d7a-1cf7-4e81-9ea9-8d7b79d5fb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:39.203894 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f9d7a-1cf7-4e81-9ea9-8d7b79d5fb31 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:39.205316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f9d7a-1cf7-4e81-9ea9-8d7b79d5fb31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:39.205692 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f9d7a-1cf7-4e81-9ea9-8d7b79d5fb31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:39.207129 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f9d7a-1cf7-4e81-9ea9-8d7b79d5fb31 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:39.208936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b14826-4b71-482b-a5c4-867009ec4a98 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:39.210477 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b14826-4b71-482b-a5c4-867009ec4a98 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:39.219384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 260/523] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:39.220549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f9d7a-1cf7-4e81-9ea9-8d7b79d5fb31 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:39.228570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-30b14826-4b71-482b-a5c4-867009ec4a98 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 12:09:39.228570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:39.228570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:39.228570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:39.228570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:39.228570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:39.228570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:39.228570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:39.228570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:39.228570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:39.228570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:39.228570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:39.228570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:09:39.228570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:09:39.228570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:09:39.228570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:39.229817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:39.230962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:39.230962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:39.230962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:09:39.230962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:09:39.230962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:09:39.230962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:09:39.230962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:09:39.230962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:09:39.230962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 12:09:39.230962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:39.236505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 264/524] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:39.236505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b14826-4b71-482b-a5c4-867009ec4a98 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:39.309991 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:39.333549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f9d7a-1cf7-4e81-9ea9-8d7b79d5fb31 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:39.339127 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b14826-4b71-482b-a5c4-867009ec4a98 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:39.368033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca624d0-698e-4de3-a09a-e1ad22c869ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:39.368723 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca624d0-698e-4de3-a09a-e1ad22c869ea None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:39.369067 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca624d0-698e-4de3-a09a-e1ad22c869ea None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:39.369445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca624d0-698e-4de3-a09a-e1ad22c869ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:39.373943 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd2e35-217d-4f8a-91b3-e9b2fb9e534a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:39.374735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd2e35-217d-4f8a-91b3-e9b2fb9e534a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:39.375182 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd2e35-217d-4f8a-91b3-e9b2fb9e534a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:39.375628 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd2e35-217d-4f8a-91b3-e9b2fb9e534a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:39.376595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd2e35-217d-4f8a-91b3-e9b2fb9e534a None admin] RBAC: Adding query filter params (role.id=60963384f3e54d9cbfbc8d12023a6cd9, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=982b82e8ac424318a934f9149fb91706) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:39.377086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd2e35-217d-4f8a-91b3-e9b2fb9e534a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:39.378696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd2e35-217d-4f8a-91b3-e9b2fb9e534a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:39.393697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 265/525] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:09:39 2022] GET /identity/v3/role_assignments?user.id=982b82e8ac424318a934f9149fb91706&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=60963384f3e54d9cbfbc8d12023a6cd9 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:39.394901 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd2e35-217d-4f8a-91b3-e9b2fb9e534a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:39.424058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:39.449763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca624d0-698e-4de3-a09a-e1ad22c869ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:39.451372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca624d0-698e-4de3-a09a-e1ad22c869ea None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:39.541075 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:39.550751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd2e35-217d-4f8a-91b3-e9b2fb9e534a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:39.569191 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 261/526] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:39.570191 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca624d0-698e-4de3-a09a-e1ad22c869ea None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:39.602109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82489dda-71c4-415e-aad8-1394ddb6a387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:39.602736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82489dda-71c4-415e-aad8-1394ddb6a387 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:39.603016 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82489dda-71c4-415e-aad8-1394ddb6a387 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:39.603296 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82489dda-71c4-415e-aad8-1394ddb6a387 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:39.612915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82489dda-71c4-415e-aad8-1394ddb6a387 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:39.614463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82489dda-71c4-415e-aad8-1394ddb6a387 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:39.635184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-82489dda-71c4-415e-aad8-1394ddb6a387 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:39.635184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:39.635184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:39.635184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:39.635184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:39.635184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:39.635184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:39.635184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:39.635184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:39.635184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:39.635184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:39.635184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:39.635184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:09:39.635184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:09:39.635184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:09:39.636086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:09:39.636086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:39.636086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:39.636086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:39.636086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:39.636086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:39.636086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:39.636086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:39.636086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:39.636086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:39.636086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:39.636086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:39.636086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:39.636086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:39.636086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:39.636086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:39.636086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:39.637014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:09:39.637014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:09:39.637014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:09:39.637014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:09:39.637014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:09:39.637014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:09:39.637014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:39.637014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:39.643431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 266/527] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:39.643431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82489dda-71c4-415e-aad8-1394ddb6a387 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:39.700082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca624d0-698e-4de3-a09a-e1ad22c869ea None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:39.745285 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57267666-167a-4787-b85b-cba804509ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:39.746246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57267666-167a-4787-b85b-cba804509ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:39.746738 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57267666-167a-4787-b85b-cba804509ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:39.747210 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57267666-167a-4787-b85b-cba804509ca9 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:39.748786 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57267666-167a-4787-b85b-cba804509ca9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:39.749428 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57267666-167a-4787-b85b-cba804509ca9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:39.751180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:39.753441 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57267666-167a-4787-b85b-cba804509ca9 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:39.778180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82489dda-71c4-415e-aad8-1394ddb6a387 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:39.789176 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 262/528] 10.222.0.60 () {56 vars in 1139 bytes} [Wed Sep 28 12:09:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:39.790424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57267666-167a-4787-b85b-cba804509ca9 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:39.819307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9d996-dad5-408b-a3ec-9e34c827edab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:39.819894 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9d996-dad5-408b-a3ec-9e34c827edab None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:39.820183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9d996-dad5-408b-a3ec-9e34c827edab None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:39.820458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9d996-dad5-408b-a3ec-9e34c827edab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:39.832178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c9d996-dad5-408b-a3ec-9e34c827edab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:39.833731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c9d996-dad5-408b-a3ec-9e34c827edab None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:39.839512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:39.850053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-95c9d996-dad5-408b-a3ec-9e34c827edab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:39.850053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:39.850053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:39.850053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:39.850053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:39.850053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:39.850053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:39.850053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:39.850053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:39.850053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:39.850053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:39.850053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:39.850053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:39.850053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:39.850053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:39.850053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:39.850996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:39.850996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:39.850996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:39.850996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:39.850996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:39.850996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:39.850996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:39.850996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:39.850996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:39.850996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:39.850996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:39.850996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:39.850996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:39.850996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:39.850996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:39.850996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:39.850996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:39.851841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:39.851841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:39.851841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:39.851841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 267/529] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:39.852056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9d996-dad5-408b-a3ec-9e34c827edab None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:39.934118 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57267666-167a-4787-b85b-cba804509ca9 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:39.970752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb20e05-061e-4bd4-96c1-c27255e70509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:39.971502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb20e05-061e-4bd4-96c1-c27255e70509 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:39.971682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb20e05-061e-4bd4-96c1-c27255e70509 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:39.971908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb20e05-061e-4bd4-96c1-c27255e70509 None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:39.972606 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb20e05-061e-4bd4-96c1-c27255e70509 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:39.972962 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb20e05-061e-4bd4-96c1-c27255e70509 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:39.974549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb20e05-061e-4bd4-96c1-c27255e70509 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:39.979090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9d996-dad5-408b-a3ec-9e34c827edab None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:39.992553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 263/530] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:09:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:39.992553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb20e05-061e-4bd4-96c1-c27255e70509 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:40.015273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:40.025291 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3d7f9-9225-4aca-ad4c-88a0e9e98b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:40.025906 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3d7f9-9225-4aca-ad4c-88a0e9e98b65 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:40.026187 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3d7f9-9225-4aca-ad4c-88a0e9e98b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:40.026461 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3d7f9-9225-4aca-ad4c-88a0e9e98b65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:40.106284 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3d7f9-9225-4aca-ad4c-88a0e9e98b65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:40.107701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3d7f9-9225-4aca-ad4c-88a0e9e98b65 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:40.128565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:40.136622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb20e05-061e-4bd4-96c1-c27255e70509 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:40.181253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2de6c-77bf-423a-b1b2-5bbbbc79e831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:40.182156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2de6c-77bf-423a-b1b2-5bbbbc79e831 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:40.182782 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2de6c-77bf-423a-b1b2-5bbbbc79e831 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:40.184375 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2de6c-77bf-423a-b1b2-5bbbbc79e831 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:40.185846 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2de6c-77bf-423a-b1b2-5bbbbc79e831 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:40.186161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2de6c-77bf-423a-b1b2-5bbbbc79e831 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:40.187545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2de6c-77bf-423a-b1b2-5bbbbc79e831 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:40.205932 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 264/531] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:40.207189 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2de6c-77bf-423a-b1b2-5bbbbc79e831 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:40.253656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 268/532] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:40.254206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3d7f9-9225-4aca-ad4c-88a0e9e98b65 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:40.294177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:40.322052 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2de6c-77bf-423a-b1b2-5bbbbc79e831 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:40.364082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b2c2d-49c9-4d49-a549-92e0dfb6a0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:40.365579 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b2c2d-49c9-4d49-a549-92e0dfb6a0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:40.366130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b2c2d-49c9-4d49-a549-92e0dfb6a0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:40.366542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b2c2d-49c9-4d49-a549-92e0dfb6a0a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:40.367655 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b2c2d-49c9-4d49-a549-92e0dfb6a0a0 None admin] RBAC: Adding query filter params (role.id=60963384f3e54d9cbfbc8d12023a6cd9, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=2354a4fd72cc42c58d6830eecb1e6253) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:40.368116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b2c2d-49c9-4d49-a549-92e0dfb6a0a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:40.370424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b2c2d-49c9-4d49-a549-92e0dfb6a0a0 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:40.385556 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 265/533] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:09:40 2022] GET /identity/v3/role_assignments?user.id=2354a4fd72cc42c58d6830eecb1e6253&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=60963384f3e54d9cbfbc8d12023a6cd9 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:40.385556 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b2c2d-49c9-4d49-a549-92e0dfb6a0a0 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:40.404820 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3d7f9-9225-4aca-ad4c-88a0e9e98b65 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:40.432315 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:40.478328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866ce22-654a-44a5-9899-3a7874a4def9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:40.478986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866ce22-654a-44a5-9899-3a7874a4def9 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:40.479271 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866ce22-654a-44a5-9899-3a7874a4def9 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:40.479543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866ce22-654a-44a5-9899-3a7874a4def9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:40.499803 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866ce22-654a-44a5-9899-3a7874a4def9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:40.503468 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866ce22-654a-44a5-9899-3a7874a4def9 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:40.528577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-3866ce22-654a-44a5-9899-3a7874a4def9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:40.528577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:40.528577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:40.528577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:40.528577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:40.528577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:40.528577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:40.528577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:40.528577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:40.528577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:40.528577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:40.528577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:40.528577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:09:40.528577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:09:40.528577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:09:40.529519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:09:40.529519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:40.529519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:40.529519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:40.529519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:40.529519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:40.529519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:40.529519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:40.529519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:40.529519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:40.529519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:40.529519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:40.529519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:40.529519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:40.529519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:40.529519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:40.529519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:40.530777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:09:40.530777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:09:40.530777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:09:40.530777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:09:40.530777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:09:40.530777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:09:40.530777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:40.530777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:40.530777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 269/534] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:40.531244 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c82a30-9171-4d05-b83d-d514d66aeedd None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:40.531244 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c82a30-9171-4d05-b83d-d514d66aeedd None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:40.531594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c82a30-9171-4d05-b83d-d514d66aeedd None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:40.537701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 270/535] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:40.540462 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c82a30-9171-4d05-b83d-d514d66aeedd None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:40.583359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b2c2d-49c9-4d49-a549-92e0dfb6a0a0 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:40.623479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149c747-0a2e-4c47-85f4-c0ba8beb1f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:40.624492 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149c747-0a2e-4c47-85f4-c0ba8beb1f0d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:40.624987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149c747-0a2e-4c47-85f4-c0ba8beb1f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:40.625496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149c747-0a2e-4c47-85f4-c0ba8beb1f0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:40.638284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8149c747-0a2e-4c47-85f4-c0ba8beb1f0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:40.640304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8149c747-0a2e-4c47-85f4-c0ba8beb1f0d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:40.647882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:40.661362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-8149c747-0a2e-4c47-85f4-c0ba8beb1f0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:40.661362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:40.661362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:40.661362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:40.661362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:40.661362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:40.661362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:40.661362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:40.661362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:40.661362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:40.661362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:40.661362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:40.661362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:40.661362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:40.661362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:40.661362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:40.662306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:40.662306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:40.662306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:40.662306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:40.662306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:40.662306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:40.662306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:40.662306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:40.662306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:40.662306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:40.662306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:40.662306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:40.662306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:40.662306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:40.662306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:40.662306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:40.662306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:40.663163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:40.663163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:40.663163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:40.665620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 266/536] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:40.666778 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c82a30-9171-4d05-b83d-d514d66aeedd None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:40.667421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff97a44-8268-4fb2-bcde-3d7c1a476664 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:40.667917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff97a44-8268-4fb2-bcde-3d7c1a476664 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:40.668647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff97a44-8268-4fb2-bcde-3d7c1a476664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:40.693781 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-fff97a44-8268-4fb2-bcde-3d7c1a476664 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:40.696026 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-fff97a44-8268-4fb2-bcde-3d7c1a476664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:40.704941 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d662f-2ab9-4ad9-8607-91997312ceed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:40.705621 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d662f-2ab9-4ad9-8607-91997312ceed None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:40.705952 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d662f-2ab9-4ad9-8607-91997312ceed None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:40.706243 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d662f-2ab9-4ad9-8607-91997312ceed None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:40.706915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d662f-2ab9-4ad9-8607-91997312ceed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:40.707274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d662f-2ab9-4ad9-8607-91997312ceed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:40.709252 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d662f-2ab9-4ad9-8607-91997312ceed None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:40.734263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 271/537] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:09:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:40.734263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d662f-2ab9-4ad9-8607-91997312ceed None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:40.800476 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 267/538] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:40.800476 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff97a44-8268-4fb2-bcde-3d7c1a476664 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:40.805584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:40.882934 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d662f-2ab9-4ad9-8607-91997312ceed None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:40.937527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80395963-3675-4e54-9587-df9e33e487e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:40.938233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80395963-3675-4e54-9587-df9e33e487e6 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:40.938474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80395963-3675-4e54-9587-df9e33e487e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:40.938743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80395963-3675-4e54-9587-df9e33e487e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:40.939378 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80395963-3675-4e54-9587-df9e33e487e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:40.939678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80395963-3675-4e54-9587-df9e33e487e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:40.942339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80395963-3675-4e54-9587-df9e33e487e6 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:40.968737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 272/539] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:40.974268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80395963-3675-4e54-9587-df9e33e487e6 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:40.993122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff97a44-8268-4fb2-bcde-3d7c1a476664 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:41.014261 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:41.053618 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3992ba-5cf8-404d-9f71-6dd5865466ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:41.054295 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3992ba-5cf8-404d-9f71-6dd5865466ee None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:41.054577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3992ba-5cf8-404d-9f71-6dd5865466ee None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:41.054848 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3992ba-5cf8-404d-9f71-6dd5865466ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:41.055574 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3992ba-5cf8-404d-9f71-6dd5865466ee None admin] RBAC: Adding query filter params (role.id=60963384f3e54d9cbfbc8d12023a6cd9, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=8e08e00bcb5f47ffab58974ba376938a) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:41.055880 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3992ba-5cf8-404d-9f71-6dd5865466ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:41.057334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3992ba-5cf8-404d-9f71-6dd5865466ee None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:41.072672 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 268/540] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:09:40 2022] GET /identity/v3/role_assignments?user.id=8e08e00bcb5f47ffab58974ba376938a&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=60963384f3e54d9cbfbc8d12023a6cd9 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:41.095061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80395963-3675-4e54-9587-df9e33e487e6 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:41.117169 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3992ba-5cf8-404d-9f71-6dd5865466ee None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:41.131688 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351900f-af08-44f6-bc76-ad618d819cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:41.132343 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351900f-af08-44f6-bc76-ad618d819cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:41.132624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351900f-af08-44f6-bc76-ad618d819cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:41.132899 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351900f-af08-44f6-bc76-ad618d819cd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:41.140866 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351900f-af08-44f6-bc76-ad618d819cd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:41.143038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351900f-af08-44f6-bc76-ad618d819cd9 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:41.155840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-4351900f-af08-44f6-bc76-ad618d819cd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:41.155840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:41.155840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:41.155840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:41.155840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:41.155840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:41.155840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:41.155840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:41.155840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:41.155840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:41.155840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:41.155840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:41.155840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:09:41.155840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:09:41.155840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:09:41.156762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:09:41.156762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:41.156762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:41.156762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:41.156762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:41.156762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:41.156762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:41.156762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:41.156762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:41.156762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:41.156762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:41.156762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:41.156762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:41.156762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:41.156762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:41.156762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:41.156762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:41.157795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:09:41.157795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:09:41.157795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:09:41.157795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:09:41.157795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:09:41.157795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:09:41.157795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:41.157795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:41.157795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 273/541] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:41.165561 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351900f-af08-44f6-bc76-ad618d819cd9 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:41.263282 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:41.304109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:41.319103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3992ba-5cf8-404d-9f71-6dd5865466ee None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:41.347252 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351900f-af08-44f6-bc76-ad618d819cd9 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:41.357932 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867b13a-55c2-405f-892c-cba5430eeb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:41.359218 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867b13a-55c2-405f-892c-cba5430eeb05 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:41.359573 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867b13a-55c2-405f-892c-cba5430eeb05 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:41.359924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867b13a-55c2-405f-892c-cba5430eeb05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:41.371035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867b13a-55c2-405f-892c-cba5430eeb05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:41.377386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-0867b13a-55c2-405f-892c-cba5430eeb05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:41.377386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:41.377386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:41.377386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:41.377386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:41.377386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:41.377386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:41.377386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:41.377386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:41.377386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:41.377386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:41.377386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:41.377386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:41.377386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:41.377386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:41.377386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:41.378459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:41.378459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:41.378459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:41.378459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:41.378459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:41.378459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:41.378459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:41.378459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:41.378459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:41.378459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:41.378459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:41.378459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:41.378459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:41.378459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:41.378459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:41.378459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:41.379437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:41.379437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:41.379437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:41.379437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:41.379437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:41.379437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:41.379437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:41.379437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:41.379437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:41.379437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:41.379437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:41.379437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:41.380907 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 269/542] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:41.386348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867b13a-55c2-405f-892c-cba5430eeb05 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:41.395797 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef333e2-0c34-4207-bfae-d2d94df16fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:41.396642 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef333e2-0c34-4207-bfae-d2d94df16fda None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:41.397107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef333e2-0c34-4207-bfae-d2d94df16fda None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:41.397594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef333e2-0c34-4207-bfae-d2d94df16fda None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:41.398450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef333e2-0c34-4207-bfae-d2d94df16fda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:41.398920 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef333e2-0c34-4207-bfae-d2d94df16fda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:41.400504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef333e2-0c34-4207-bfae-d2d94df16fda None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:41.431179 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 274/543] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:09:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:41.440598 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef333e2-0c34-4207-bfae-d2d94df16fda None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:41.575319 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef333e2-0c34-4207-bfae-d2d94df16fda None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:41.605529 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:41.615917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9553ed-c93f-4043-85b3-ee387723acb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:41.616540 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9553ed-c93f-4043-85b3-ee387723acb0 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:41.617283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9553ed-c93f-4043-85b3-ee387723acb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:41.617283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9553ed-c93f-4043-85b3-ee387723acb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:41.617838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9553ed-c93f-4043-85b3-ee387723acb0 None admin] RBAC: Adding query filter params (role.id=60963384f3e54d9cbfbc8d12023a6cd9, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=ba75586d3a7d4278ad6c0c31a1bed3df) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:41.618147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9553ed-c93f-4043-85b3-ee387723acb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:41.619652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9553ed-c93f-4043-85b3-ee387723acb0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:41.635701 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867b13a-55c2-405f-892c-cba5430eeb05 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:41.639355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 275/544] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:09:41 2022] GET /identity/v3/role_assignments?user.id=ba75586d3a7d4278ad6c0c31a1bed3df&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=60963384f3e54d9cbfbc8d12023a6cd9 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:41.669877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:41.679984 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f005a-c563-490b-b830-2825aee041f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:41.680994 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f005a-c563-490b-b830-2825aee041f7 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:41.681507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f005a-c563-490b-b830-2825aee041f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:41.681993 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f005a-c563-490b-b830-2825aee041f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:41.683211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f005a-c563-490b-b830-2825aee041f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:41.683767 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f005a-c563-490b-b830-2825aee041f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:41.686490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f005a-c563-490b-b830-2825aee041f7 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:41.698864 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 270/545] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:09:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:41.704751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9553ed-c93f-4043-85b3-ee387723acb0 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:41.768224 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35699ea-9943-4129-a671-1f1f0f074a8a None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:41.768539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35699ea-9943-4129-a671-1f1f0f074a8a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:41.768766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35699ea-9943-4129-a671-1f1f0f074a8a None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:41.770182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 271/546] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:09:41.778341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e464c7b-76d0-481b-b8ff-971a41fe8a83 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:41.778643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e464c7b-76d0-481b-b8ff-971a41fe8a83 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:41.778882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e464c7b-76d0-481b-b8ff-971a41fe8a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:41.811145 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-9e464c7b-76d0-481b-b8ff-971a41fe8a83 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:41.812382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-9e464c7b-76d0-481b-b8ff-971a41fe8a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:41.865261 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9553ed-c93f-4043-85b3-ee387723acb0 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:41.909842 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59782bab-63cd-41e9-9dee-64ed8a78323d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:41.910478 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59782bab-63cd-41e9-9dee-64ed8a78323d None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:41.910753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59782bab-63cd-41e9-9dee-64ed8a78323d None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:41.911020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59782bab-63cd-41e9-9dee-64ed8a78323d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:41.933004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59782bab-63cd-41e9-9dee-64ed8a78323d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:41.935687 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59782bab-63cd-41e9-9dee-64ed8a78323d None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:41.940414 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 272/547] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:09:41.953841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-59782bab-63cd-41e9-9dee-64ed8a78323d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:41.953841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:41.953841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:41.953841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:41.953841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:41.953841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:41.953841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:41.953841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:41.953841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:41.953841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:41.953841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:41.953841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:41.953841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:41.953841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:41.953841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:41.953841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:41.954851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:41.954851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:41.954851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:41.954851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:41.954851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:41.954851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:41.954851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:41.954851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:41.954851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:41.954851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:41.954851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:41.954851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:41.954851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:41.954851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:41.954851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:41.954851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:41.954851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:41.955687 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:41.955687 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:41.955687 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:41.955687 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 276/548] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:41.962190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e464c7b-76d0-481b-b8ff-971a41fe8a83 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:42.095368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e464c7b-76d0-481b-b8ff-971a41fe8a83 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:42.122961 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:42.146742 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c847-7fb2-423f-a60a-6b8f933b2139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:42.147355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c847-7fb2-423f-a60a-6b8f933b2139 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:42.147637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c847-7fb2-423f-a60a-6b8f933b2139 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:42.147909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1c847-7fb2-423f-a60a-6b8f933b2139 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:42.148551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1c847-7fb2-423f-a60a-6b8f933b2139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:42.148852 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1c847-7fb2-423f-a60a-6b8f933b2139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:42.150328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c1c847-7fb2-423f-a60a-6b8f933b2139 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:42.166928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 273/549] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:42.172216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59782bab-63cd-41e9-9dee-64ed8a78323d None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:42.227087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:42.284117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59782bab-63cd-41e9-9dee-64ed8a78323d None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:42.318423 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d8ec7-e563-4486-bf51-35efcc668395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:42.319188 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d8ec7-e563-4486-bf51-35efcc668395 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:42.319384 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d8ec7-e563-4486-bf51-35efcc668395 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:42.319662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d8ec7-e563-4486-bf51-35efcc668395 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:42.327335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d8ec7-e563-4486-bf51-35efcc668395 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:42.329312 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d8ec7-e563-4486-bf51-35efcc668395 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:42.331235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c847-7fb2-423f-a60a-6b8f933b2139 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:42.343915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-4e7d8ec7-e563-4486-bf51-35efcc668395 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 12:09:42.343915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:42.343915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:42.343915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:42.343915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:42.343915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:42.343915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:42.343915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:42.343915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:42.343915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:42.343915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:42.343915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:42.343915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:09:42.343915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:09:42.343915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:09:42.343915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:42.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:42.347727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:42.347727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:42.347727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:09:42.347727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:09:42.347727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:09:42.347727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:09:42.347727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:09:42.347727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:09:42.347727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 12:09:42.347727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:42.347727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 277/550] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:09:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:42.352017 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d8ec7-e563-4486-bf51-35efcc668395 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:42.502512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c1c847-7fb2-423f-a60a-6b8f933b2139 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:42.571368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b79a98-8910-4636-9346-a7510255212a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:42.572005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b79a98-8910-4636-9346-a7510255212a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:42.572286 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b79a98-8910-4636-9346-a7510255212a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:42.572558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b79a98-8910-4636-9346-a7510255212a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:42.580051 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d8ec7-e563-4486-bf51-35efcc668395 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:42.593506 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:42.602134 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b79a98-8910-4636-9346-a7510255212a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:42.608103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-21b79a98-8910-4636-9346-a7510255212a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:42.608103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:42.608103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:42.608103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:42.608103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:42.608103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:42.608103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:42.608103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:42.608103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:42.608103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:42.608103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:42.608103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:42.608103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:42.608103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:42.608103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:42.608103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:42.609097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:42.609097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:42.609097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:42.609097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:42.609097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:42.609097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:42.609097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:42.609097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:42.609097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:42.609097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:42.609097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:42.609097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:42.609097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:42.609097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:42.609097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:42.609097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:42.610344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:42.610344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:42.610344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:42.610344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:42.610344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:42.610344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:42.610344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:42.610344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:42.610344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:42.610344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:42.610344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:42.610344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:42.619333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 274/551] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:42.622958 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bbe27-95be-4d72-abf6-3c85cccac7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:42.623533 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bbe27-95be-4d72-abf6-3c85cccac7cf None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:42.623813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bbe27-95be-4d72-abf6-3c85cccac7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:42.624470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bbe27-95be-4d72-abf6-3c85cccac7cf None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:42.625208 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b79a98-8910-4636-9346-a7510255212a None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:42.625331 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bbe27-95be-4d72-abf6-3c85cccac7cf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:42.625644 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bbe27-95be-4d72-abf6-3c85cccac7cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:42.627082 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bbe27-95be-4d72-abf6-3c85cccac7cf None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:42.662190 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 278/552] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Sep 28 12:09:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:42.669800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bbe27-95be-4d72-abf6-3c85cccac7cf None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:42.682136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:42.806845 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b79a98-8910-4636-9346-a7510255212a None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:42.829868 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bbe27-95be-4d72-abf6-3c85cccac7cf None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:42.884382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7c95f-cebb-4c0b-b090-49357634d78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:42.885000 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7c95f-cebb-4c0b-b090-49357634d78f None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:42.885313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7c95f-cebb-4c0b-b090-49357634d78f None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:42.885591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7c95f-cebb-4c0b-b090-49357634d78f None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:42.886264 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d7c95f-cebb-4c0b-b090-49357634d78f None admin] RBAC: Adding query filter params (name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:42.886575 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d7c95f-cebb-4c0b-b090-49357634d78f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:42.887947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d7c95f-cebb-4c0b-b090-49357634d78f None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:42.899703 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98642b-72a7-4a1b-8ecb-382b2e945852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:42.900310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98642b-72a7-4a1b-8ecb-382b2e945852 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:42.900593 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98642b-72a7-4a1b-8ecb-382b2e945852 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:42.900866 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98642b-72a7-4a1b-8ecb-382b2e945852 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:42.911299 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 275/553] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:09:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:42.916971 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7c95f-cebb-4c0b-b090-49357634d78f None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:42.962686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:43.015681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98642b-72a7-4a1b-8ecb-382b2e945852 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:43.018028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98642b-72a7-4a1b-8ecb-382b2e945852 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:43.038949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:43.043224 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7c95f-cebb-4c0b-b090-49357634d78f None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:43.092325 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058eeb7-8fc7-4fc1-a6f0-907e7607ed17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:43.093161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058eeb7-8fc7-4fc1-a6f0-907e7607ed17 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:43.093452 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058eeb7-8fc7-4fc1-a6f0-907e7607ed17 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:43.093729 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058eeb7-8fc7-4fc1-a6f0-907e7607ed17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:43.109004 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9058eeb7-8fc7-4fc1-a6f0-907e7607ed17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:43.111318 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9058eeb7-8fc7-4fc1-a6f0-907e7607ed17 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:43.133512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-9058eeb7-8fc7-4fc1-a6f0-907e7607ed17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:43.133512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:43.133512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:43.133512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:43.133512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:43.133512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:43.133512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:43.133512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:43.133512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:43.133512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:43.133512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:43.133512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:43.133512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:43.133512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:43.133512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:43.133512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:43.134473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:43.134473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:43.134473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:43.134473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:43.134473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:43.134473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:43.134473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:43.134473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:43.134473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:43.134473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:43.134473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:43.134473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:43.134473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:43.134473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:43.134473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:43.134473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:43.134473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:43.135351 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:43.135351 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:43.135351 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:43.137213 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 276/554] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:43.138786 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69ede9-8312-4ed2-856e-f8d5e3d21daa None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:43.139259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69ede9-8312-4ed2-856e-f8d5e3d21daa None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:43.140508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69ede9-8312-4ed2-856e-f8d5e3d21daa None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:43.146326 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 277/555] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:43.147333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69ede9-8312-4ed2-856e-f8d5e3d21daa None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:43.148378 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 279/556] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:43.149845 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd73fde-3a6a-4c3c-80c8-a804be72d7e1 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:43.150135 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd73fde-3a6a-4c3c-80c8-a804be72d7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:43.150420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd73fde-3a6a-4c3c-80c8-a804be72d7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:43.177505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-fdd73fde-3a6a-4c3c-80c8-a804be72d7e1 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:43.181259 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-fdd73fde-3a6a-4c3c-80c8-a804be72d7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:43.247994 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:43.267650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 280/557] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:43.268848 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd73fde-3a6a-4c3c-80c8-a804be72d7e1 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:43.310752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69ede9-8312-4ed2-856e-f8d5e3d21daa None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:43.352009 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9802780-602e-4fb0-b66c-18d80e5d5fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:43.352921 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9802780-602e-4fb0-b66c-18d80e5d5fdf None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:43.353242 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9802780-602e-4fb0-b66c-18d80e5d5fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:43.353520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9802780-602e-4fb0-b66c-18d80e5d5fdf None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:43.354166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9802780-602e-4fb0-b66c-18d80e5d5fdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:43.354471 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9802780-602e-4fb0-b66c-18d80e5d5fdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:43.355866 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9802780-602e-4fb0-b66c-18d80e5d5fdf None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:43.371168 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 278/558] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:43.377061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9802780-602e-4fb0-b66c-18d80e5d5fdf None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:43.394637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:43.493467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9802780-602e-4fb0-b66c-18d80e5d5fdf None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:43.534178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1e027-318a-4f0b-9bd3-218c9250dba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:43.534942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1e027-318a-4f0b-9bd3-218c9250dba1 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:43.535457 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1e027-318a-4f0b-9bd3-218c9250dba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:43.535744 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1e027-318a-4f0b-9bd3-218c9250dba1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:43.543740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1e027-318a-4f0b-9bd3-218c9250dba1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:43.545319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1e027-318a-4f0b-9bd3-218c9250dba1 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:43.547933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd73fde-3a6a-4c3c-80c8-a804be72d7e1 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:43.559678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-04e1e027-318a-4f0b-9bd3-218c9250dba1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:09:43.559678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:43.559678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:43.559678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:43.559678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:43.559678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:43.559678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:43.559678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:43.559678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:43.559678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:43.559678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:43.559678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:43.559678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:09:43.559678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:09:43.559678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:09:43.559678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:43.560657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:43.561625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:43.561625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:43.561625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:09:43.561625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:09:43.561625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:09:43.561625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:09:43.561625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:09:43.561625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:09:43.561625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:09:43.561625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:43.561625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 279/559] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 12:09:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:43.568123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1e027-318a-4f0b-9bd3-218c9250dba1 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:43.604191 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbd09d-ea72-4067-9120-5d9632254cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:43.604736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd09d-ea72-4067-9120-5d9632254cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:43.605013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd09d-ea72-4067-9120-5d9632254cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:43.605310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd09d-ea72-4067-9120-5d9632254cc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:43.618525 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbd09d-ea72-4067-9120-5d9632254cc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:43.619957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbd09d-ea72-4067-9120-5d9632254cc1 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:43.637981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-5cdbd09d-ea72-4067-9120-5d9632254cc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:43.637981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:43.637981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:43.637981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:43.637981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:43.637981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:43.637981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:43.637981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:43.637981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:43.637981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:43.637981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:43.637981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:43.637981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:43.637981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:43.637981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:43.637981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:43.639066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:43.639066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:43.639066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:43.639066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:43.639066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:43.639066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:43.639066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:43.639066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:43.639066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:43.639066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:43.639066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:43.639066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:43.639066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:43.639066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:43.639066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:43.639066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:43.639066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:43.639962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:43.639962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:43.639962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:43.639962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 281/560] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:43.640185 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbd09d-ea72-4067-9120-5d9632254cc1 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:43.654758 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:43.698039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:43.699867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1e027-318a-4f0b-9bd3-218c9250dba1 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:43.749150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331a38c-18f6-494d-b4da-1ac415be975a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:43.750017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331a38c-18f6-494d-b4da-1ac415be975a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:43.750448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331a38c-18f6-494d-b4da-1ac415be975a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:43.751410 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331a38c-18f6-494d-b4da-1ac415be975a None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:43.752291 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331a38c-18f6-494d-b4da-1ac415be975a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:43.752751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331a38c-18f6-494d-b4da-1ac415be975a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:43.754917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331a38c-18f6-494d-b4da-1ac415be975a None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:43.767005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbd09d-ea72-4067-9120-5d9632254cc1 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:43.783119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 280/561] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 12:09:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:43.783119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331a38c-18f6-494d-b4da-1ac415be975a None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:43.817037 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232fc58-ddbb-4da2-a12c-024a12ff186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:43.817749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232fc58-ddbb-4da2-a12c-024a12ff186f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:43.818025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232fc58-ddbb-4da2-a12c-024a12ff186f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:43.818397 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232fc58-ddbb-4da2-a12c-024a12ff186f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:43.832471 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232fc58-ddbb-4da2-a12c-024a12ff186f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:43.838026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-5232fc58-ddbb-4da2-a12c-024a12ff186f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:43.838026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:43.838026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:43.838026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:43.838026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:43.838026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:43.838026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:43.838026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:43.838026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:43.838026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:43.838026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:43.838026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:43.838026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:43.838026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:43.838026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:43.838026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:43.839134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:43.839134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:43.839134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:43.839134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:43.839134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:43.839134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:43.839134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:43.839134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:43.839134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:43.839134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:43.839134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:43.839134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:43.839134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:43.839134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:43.839134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:43.839134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:43.839978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:43.839978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:43.839978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:43.839978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:43.839978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:43.839978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:43.839978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:43.839978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:43.839978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:43.839978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:43.839978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:43.839978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:43.839978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 282/562] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:43.840651 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232fc58-ddbb-4da2-a12c-024a12ff186f None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:43.906723 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331a38c-18f6-494d-b4da-1ac415be975a None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:43.937341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:43.944380 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80436496-8b0e-44fb-883a-bea329e17401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:43.944991 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80436496-8b0e-44fb-883a-bea329e17401 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:43.945294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80436496-8b0e-44fb-883a-bea329e17401 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:43.945724 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80436496-8b0e-44fb-883a-bea329e17401 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:43.946473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80436496-8b0e-44fb-883a-bea329e17401 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:43.946579 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80436496-8b0e-44fb-883a-bea329e17401 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:43.947939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80436496-8b0e-44fb-883a-bea329e17401 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:43.948636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232fc58-ddbb-4da2-a12c-024a12ff186f None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:43.951195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:43.959740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 281/563] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:43.961146 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93877a24-c8a6-4abc-a0a2-0b366f9531a0 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:43.961451 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93877a24-c8a6-4abc-a0a2-0b366f9531a0 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:43.961720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93877a24-c8a6-4abc-a0a2-0b366f9531a0 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:43.962886 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 282/564] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:43.963563 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93877a24-c8a6-4abc-a0a2-0b366f9531a0 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:43.988224 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0babe5-b51a-4fa0-a143-c3071f8024ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:43.988849 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0babe5-b51a-4fa0-a143-c3071f8024ba None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:43.989141 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0babe5-b51a-4fa0-a143-c3071f8024ba None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:43.989427 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0babe5-b51a-4fa0-a143-c3071f8024ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:43.999666 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0babe5-b51a-4fa0-a143-c3071f8024ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:44.001952 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0babe5-b51a-4fa0-a143-c3071f8024ba None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:44.015670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-af0babe5-b51a-4fa0-a143-c3071f8024ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:44.015670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:44.015670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:44.015670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:44.015670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:44.015670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:44.015670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:44.015670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:44.015670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:44.015670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:44.015670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:44.015670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:44.015670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:44.015670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:44.015670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:44.015670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:44.016671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:44.016671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:44.016671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:44.016671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:44.016671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:44.016671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:44.016671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:44.016671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:44.016671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:44.016671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:44.016671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:44.016671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:44.016671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:44.016671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:44.016671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:44.016671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:44.016671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:44.017566 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:44.017566 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:44.017566 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:44.017566 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 283/565] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:44.018223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec868f-e941-4e2f-a72c-ea5a7abec868 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:44.018507 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec868f-e941-4e2f-a72c-ea5a7abec868 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:44.018789 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec868f-e941-4e2f-a72c-ea5a7abec868 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:44.020130 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 284/566] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:44.024865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec868f-e941-4e2f-a72c-ea5a7abec868 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:44.067215 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93877a24-c8a6-4abc-a0a2-0b366f9531a0 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:44.100234 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fe921-dcba-4753-a115-bbdacc5b500c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:44.100832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fe921-dcba-4753-a115-bbdacc5b500c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:44.101111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fe921-dcba-4753-a115-bbdacc5b500c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:44.101408 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fe921-dcba-4753-a115-bbdacc5b500c None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:44.102057 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fe921-dcba-4753-a115-bbdacc5b500c None admin] RBAC: Adding query filter params (name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:44.102354 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fe921-dcba-4753-a115-bbdacc5b500c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:44.103714 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fe921-dcba-4753-a115-bbdacc5b500c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:44.113757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 283/567] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:09:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:44.114945 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017d72f-51a9-47e0-81d3-7f39505ff167 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:44.115228 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017d72f-51a9-47e0-81d3-7f39505ff167 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:44.115496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017d72f-51a9-47e0-81d3-7f39505ff167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:44.126202 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec868f-e941-4e2f-a72c-ea5a7abec868 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:44.133023 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-d017d72f-51a9-47e0-81d3-7f39505ff167 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:44.134234 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d017d72f-51a9-47e0-81d3-7f39505ff167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:44.160144 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868946ea-823f-4fd6-80fd-26a9c4577ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:44.160732 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868946ea-823f-4fd6-80fd-26a9c4577ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:44.161017 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868946ea-823f-4fd6-80fd-26a9c4577ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:44.161317 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868946ea-823f-4fd6-80fd-26a9c4577ed9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:44.167951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868946ea-823f-4fd6-80fd-26a9c4577ed9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:44.169416 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868946ea-823f-4fd6-80fd-26a9c4577ed9 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:44.181180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-868946ea-823f-4fd6-80fd-26a9c4577ed9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:44.181180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:44.181180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:44.181180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:44.181180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:44.181180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:44.181180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:44.181180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:44.181180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:44.181180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:44.181180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:44.181180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:44.181180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:09:44.181180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:09:44.181180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:09:44.182149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:09:44.182149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:44.182149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:44.182149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:44.182149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:44.182149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:44.182149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:44.182149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:44.182149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:44.182149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:44.182149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:44.182149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:44.182149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:44.182149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:44.182149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:44.182149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:44.182149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:44.183027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:09:44.183027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:09:44.183027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:09:44.183027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:09:44.183027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:09:44.183027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:09:44.183027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:44.183027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:44.183027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 285/568] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:44.183027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868946ea-823f-4fd6-80fd-26a9c4577ed9 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:44.186924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:44.203335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:44.206924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 284/569] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:44.208144 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c81ef-9888-402a-97ce-479f4d4cea3b None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:44.208429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c81ef-9888-402a-97ce-479f4d4cea3b None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:44.208701 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c81ef-9888-402a-97ce-479f4d4cea3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:44.240290 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-909c81ef-9888-402a-97ce-479f4d4cea3b None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:44.241783 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-909c81ef-9888-402a-97ce-479f4d4cea3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:44.294194 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868946ea-823f-4fd6-80fd-26a9c4577ed9 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:44.313321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 285/570] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:44.313321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c81ef-9888-402a-97ce-479f4d4cea3b None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:44.333234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30204c-6c2d-4220-8e60-cdf7e50f013d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:44.333891 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30204c-6c2d-4220-8e60-cdf7e50f013d None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:44.334166 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30204c-6c2d-4220-8e60-cdf7e50f013d None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:44.334485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30204c-6c2d-4220-8e60-cdf7e50f013d None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:44.335125 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30204c-6c2d-4220-8e60-cdf7e50f013d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:44.335424 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30204c-6c2d-4220-8e60-cdf7e50f013d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:44.336893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30204c-6c2d-4220-8e60-cdf7e50f013d None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:44.350279 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 286/571] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:44.355480 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30204c-6c2d-4220-8e60-cdf7e50f013d None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:44.438753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:44.457099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c81ef-9888-402a-97ce-479f4d4cea3b None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:44.462504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30204c-6c2d-4220-8e60-cdf7e50f013d None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:44.477828 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:44.499764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9502d-98fe-43d3-ae93-1570c8203a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:44.500362 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9502d-98fe-43d3-ae93-1570c8203a30 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:44.501335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9502d-98fe-43d3-ae93-1570c8203a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:44.501612 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9502d-98fe-43d3-ae93-1570c8203a30 None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:44.502234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9502d-98fe-43d3-ae93-1570c8203a30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:44.502533 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9502d-98fe-43d3-ae93-1570c8203a30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:44.503938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9502d-98fe-43d3-ae93-1570c8203a30 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:44.504213 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d17447-89cf-44bf-a6f8-3d812f64073e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:44.504847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d17447-89cf-44bf-a6f8-3d812f64073e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:44.505151 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d17447-89cf-44bf-a6f8-3d812f64073e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:44.505469 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d17447-89cf-44bf-a6f8-3d812f64073e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:44.517217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 287/572] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:09:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:44.517217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9502d-98fe-43d3-ae93-1570c8203a30 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:44.518256 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d17447-89cf-44bf-a6f8-3d812f64073e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:44.519735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d17447-89cf-44bf-a6f8-3d812f64073e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:44.535497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-e7d17447-89cf-44bf-a6f8-3d812f64073e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:44.535497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:44.535497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:44.535497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:44.535497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:44.535497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:44.535497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:44.535497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:44.535497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:44.535497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:44.535497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:44.535497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:44.535497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:44.535497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:44.535497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:44.535497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:44.537049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:44.537049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:44.537049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:44.537049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:44.537049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:44.537049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:44.537049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:44.537049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:44.537049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:44.537049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:44.537049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:44.537049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:44.537049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:44.537049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:44.537049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:44.537049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:44.537049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:44.538044 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:44.538044 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:44.538044 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:44.538044 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 286/573] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:44.538044 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d17447-89cf-44bf-a6f8-3d812f64073e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:44.623561 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9502d-98fe-43d3-ae93-1570c8203a30 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:44.645651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d17447-89cf-44bf-a6f8-3d812f64073e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:44.656385 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d05b8-9708-4a20-8704-6abcd7cbbab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:44.657049 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d05b8-9708-4a20-8704-6abcd7cbbab9 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:44.657360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d05b8-9708-4a20-8704-6abcd7cbbab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:44.657632 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d05b8-9708-4a20-8704-6abcd7cbbab9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:44.660203 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:44.664982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d05b8-9708-4a20-8704-6abcd7cbbab9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:44.666583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d05b8-9708-4a20-8704-6abcd7cbbab9 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:44.678635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-f49d05b8-9708-4a20-8704-6abcd7cbbab9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:44.678635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:44.678635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:44.678635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:44.678635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:44.678635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:44.678635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:44.678635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:44.678635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:44.678635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:44.678635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:44.678635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:44.678635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:09:44.678635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:09:44.678635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:09:44.679510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:09:44.679510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:44.679510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:44.679510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:44.679510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:44.679510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:44.679510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:44.679510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:44.679510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:44.679510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:44.679510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:44.679510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:44.679510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:44.679510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:44.679510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:44.679510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:44.679510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:44.680396 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:09:44.680396 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:09:44.680396 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:09:44.680396 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:09:44.680396 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:09:44.680396 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:09:44.680396 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:44.680396 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:44.680821 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac644b0b-5472-4826-a858-567370fd2263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:44.680821 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac644b0b-5472-4826-a858-567370fd2263 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:44.680965 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 288/574] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:44.681040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac644b0b-5472-4826-a858-567370fd2263 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:44.681040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac644b0b-5472-4826-a858-567370fd2263 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:44.681202 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d05b8-9708-4a20-8704-6abcd7cbbab9 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:44.695022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac644b0b-5472-4826-a858-567370fd2263 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:44.700193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-ac644b0b-5472-4826-a858-567370fd2263 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:44.700193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:44.700193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:44.700193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:44.700193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:44.700193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:44.700193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:44.700193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:44.700193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:44.700193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:44.700193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:44.700193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:44.700193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:44.700193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:44.700193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:44.700193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:44.701084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:44.701084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:44.701084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:44.701084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:44.701084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:44.701084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:44.701084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:44.701084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:44.701084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:44.701084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:44.701084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:44.701084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:44.701084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:44.701084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:44.701084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:44.701084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:44.701978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:44.701978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:44.701978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:44.701978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:44.701978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:44.701978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:44.701978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:44.701978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:44.701978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:44.701978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:44.701978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:44.701978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:44.701978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 287/575] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:44.703284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac644b0b-5472-4826-a858-567370fd2263 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:44.726796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:44.785920 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d05b8-9708-4a20-8704-6abcd7cbbab9 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:44.803121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac644b0b-5472-4826-a858-567370fd2263 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:44.820392 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a15c2-ea16-4bb9-8584-d77e15b5fe5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:44.820997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a15c2-ea16-4bb9-8584-d77e15b5fe5f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:44.821312 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a15c2-ea16-4bb9-8584-d77e15b5fe5f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:44.821586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a15c2-ea16-4bb9-8584-d77e15b5fe5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:44.822304 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a15c2-ea16-4bb9-8584-d77e15b5fe5f None admin] RBAC: Adding query filter params (role.id=60963384f3e54d9cbfbc8d12023a6cd9, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=975a8d7349994520bcbfb5f75e52aac7) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:44.822608 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a15c2-ea16-4bb9-8584-d77e15b5fe5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:44.824059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a15c2-ea16-4bb9-8584-d77e15b5fe5f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:44.829538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 289/576] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:09:44 2022] GET /identity/v3/role_assignments?user.id=975a8d7349994520bcbfb5f75e52aac7&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=60963384f3e54d9cbfbc8d12023a6cd9 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:44.830317 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a15c2-ea16-4bb9-8584-d77e15b5fe5f None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:44.836765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46da89-6743-4ae8-b2e8-cd1b2281a5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:44.837422 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46da89-6743-4ae8-b2e8-cd1b2281a5ad None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:44.837701 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46da89-6743-4ae8-b2e8-cd1b2281a5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:44.837972 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46da89-6743-4ae8-b2e8-cd1b2281a5ad None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:44.838628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46da89-6743-4ae8-b2e8-cd1b2281a5ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:44.838928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46da89-6743-4ae8-b2e8-cd1b2281a5ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:44.840383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46da89-6743-4ae8-b2e8-cd1b2281a5ad None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:44.858965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 288/577] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:44.858965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46da89-6743-4ae8-b2e8-cd1b2281a5ad None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:44.907627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:44.947867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:44.956377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a15c2-ea16-4bb9-8584-d77e15b5fe5f None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:44.964701 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46da89-6743-4ae8-b2e8-cd1b2281a5ad None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:44.997931 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b35ea-5f69-4c7b-9531-7d28e4cf8218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:44.998574 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b35ea-5f69-4c7b-9531-7d28e4cf8218 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:44.998851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b35ea-5f69-4c7b-9531-7d28e4cf8218 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:44.999116 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b35ea-5f69-4c7b-9531-7d28e4cf8218 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:45.005147 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a619c62-ec53-4a9a-9cb0-30418478bf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:45.005773 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a619c62-ec53-4a9a-9cb0-30418478bf72 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:45.006053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a619c62-ec53-4a9a-9cb0-30418478bf72 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:45.006324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a619c62-ec53-4a9a-9cb0-30418478bf72 None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:45.007019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a619c62-ec53-4a9a-9cb0-30418478bf72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:45.007325 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a619c62-ec53-4a9a-9cb0-30418478bf72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:45.008761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a619c62-ec53-4a9a-9cb0-30418478bf72 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:45.010857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b35ea-5f69-4c7b-9531-7d28e4cf8218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:45.016124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-820b35ea-5f69-4c7b-9531-7d28e4cf8218 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:45.016124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:45.016124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:45.016124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:45.016124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:45.016124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:45.016124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:45.016124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:45.016124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:45.016124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:45.016124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:45.016124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:45.016124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:45.016124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:45.016124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:45.016124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:45.017151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:45.017151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:45.017151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:45.017151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:45.017151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:45.017151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:45.017151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:45.017151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:45.017151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:45.017151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:45.017151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:45.017151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:45.017151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:45.017151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:45.017151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:45.017151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:45.018025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:45.018025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:45.018025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:45.018025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:45.018025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:45.018025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:45.018025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:45.018025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:45.018025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:45.018025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:45.018025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:45.018025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:45.021421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 290/578] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:45.022370 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b35ea-5f69-4c7b-9531-7d28e4cf8218 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:45.033662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 289/579] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:09:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:45.035139 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a619c62-ec53-4a9a-9cb0-30418478bf72 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:45.132549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b35ea-5f69-4c7b-9531-7d28e4cf8218 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:45.143306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a619c62-ec53-4a9a-9cb0-30418478bf72 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:45.153117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:45.173465 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68536e-a467-4481-9b7c-70befa06491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:45.174215 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68536e-a467-4481-9b7c-70befa06491f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:45.174497 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68536e-a467-4481-9b7c-70befa06491f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:45.174933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68536e-a467-4481-9b7c-70befa06491f None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:45.175917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68536e-a467-4481-9b7c-70befa06491f None admin] RBAC: Adding query filter params (name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:45.176232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68536e-a467-4481-9b7c-70befa06491f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:45.178087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68536e-a467-4481-9b7c-70befa06491f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:45.183775 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ff4a5-7456-4ca1-938a-7c58fee837e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:45.184481 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ff4a5-7456-4ca1-938a-7c58fee837e7 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:45.184824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ff4a5-7456-4ca1-938a-7c58fee837e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:45.185179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ff4a5-7456-4ca1-938a-7c58fee837e7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:45.194493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 291/580] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:09:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:45.194493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68536e-a467-4481-9b7c-70befa06491f None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:45.194752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ff4a5-7456-4ca1-938a-7c58fee837e7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:45.195312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ff4a5-7456-4ca1-938a-7c58fee837e7 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:45.206968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-702ff4a5-7456-4ca1-938a-7c58fee837e7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 12:09:45.206968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:45.206968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:45.206968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:45.206968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:45.206968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:45.206968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:45.206968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:45.206968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:45.206968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:45.206968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:45.206968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:45.206968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:09:45.206968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:09:45.206968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:09:45.206968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:45.208068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:45.209682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:45.209682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:45.209682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:09:45.209682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:09:45.209682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:09:45.209682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:09:45.209682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:09:45.209682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:09:45.209682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 12:09:45.209682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:45.209682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 290/581] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:45.213240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ff4a5-7456-4ca1-938a-7c58fee837e7 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:45.216768 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:45.307922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68536e-a467-4481-9b7c-70befa06491f None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:45.319591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ff4a5-7456-4ca1-938a-7c58fee837e7 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:45.342871 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3dd33-e4e4-45ac-98a7-773f11367632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:45.343647 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3dd33-e4e4-45ac-98a7-773f11367632 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:45.344329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3dd33-e4e4-45ac-98a7-773f11367632 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:45.345277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3dd33-e4e4-45ac-98a7-773f11367632 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:45.346583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3dd33-e4e4-45ac-98a7-773f11367632 None admin] RBAC: Adding query filter params (name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:45.347119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3dd33-e4e4-45ac-98a7-773f11367632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:45.349764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3dd33-e4e4-45ac-98a7-773f11367632 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:45.355571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1db31-c863-487a-81a3-cd1b132f4fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:45.356416 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1db31-c863-487a-81a3-cd1b132f4fa1 None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:45.356866 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1db31-c863-487a-81a3-cd1b132f4fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:45.357316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1db31-c863-487a-81a3-cd1b132f4fa1 None admin] PATH_INFO: `/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/982b82e8ac424318a934f9149fb91706/roles/60963384f3e54d9cbfbc8d12023a6cd9` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:45.357807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 292/582] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:09:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:45.358982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3dd33-e4e4-45ac-98a7-773f11367632 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:45.376139 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1db31-c863-487a-81a3-cd1b132f4fa1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6335d0db1f4844c39c255909189e0ce9, user_id=982b82e8ac424318a934f9149fb91706, role_id=60963384f3e54d9cbfbc8d12023a6cd9)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:45.377965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1db31-c863-487a-81a3-cd1b132f4fa1 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:45.394527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 291/583] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:09:45 2022] PUT /identity/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/982b82e8ac424318a934f9149fb91706/roles/60963384f3e54d9cbfbc8d12023a6cd9 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:09:45.395474 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1db31-c863-487a-81a3-cd1b132f4fa1 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:45.414857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:45.463791 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:45.489215 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3dd33-e4e4-45ac-98a7-773f11367632 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:45.521120 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1db31-c863-487a-81a3-cd1b132f4fa1 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:45.536082 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94967c3a-d6f9-4c1b-8a85-544ff79ff426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:45.537223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94967c3a-d6f9-4c1b-8a85-544ff79ff426 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:45.537712 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94967c3a-d6f9-4c1b-8a85-544ff79ff426 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:45.538237 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94967c3a-d6f9-4c1b-8a85-544ff79ff426 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:45.554114 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94967c3a-d6f9-4c1b-8a85-544ff79ff426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:45.556903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94967c3a-d6f9-4c1b-8a85-544ff79ff426 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:45.565053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda06fc9-230d-42f4-843b-9939e335d00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:45.565617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda06fc9-230d-42f4-843b-9939e335d00d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:45.565779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda06fc9-230d-42f4-843b-9939e335d00d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:45.566196 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda06fc9-230d-42f4-843b-9939e335d00d None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:45.567006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda06fc9-230d-42f4-843b-9939e335d00d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:45.567383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda06fc9-230d-42f4-843b-9939e335d00d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:45.569017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda06fc9-230d-42f4-843b-9939e335d00d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:45.576119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-94967c3a-d6f9-4c1b-8a85-544ff79ff426 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:45.576119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:45.576119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:45.576119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:45.576119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:45.576119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:45.576119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:45.576119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:45.576119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:45.576119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:45.576119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:45.576119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:45.576119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:45.576119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:45.576119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:45.576119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:45.577090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:45.577090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:45.577090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:45.577090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:45.577090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:45.577090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:45.577090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:45.577090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:45.577090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:45.577090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:45.577090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:45.577090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:45.577090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:45.577090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:45.577090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:45.577090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:45.577090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:45.578024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:45.578024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:45.578024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:45.578024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 293/584] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:45.579443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94967c3a-d6f9-4c1b-8a85-544ff79ff426 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:45.598613 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 292/585] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Sep 28 12:09:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:45.599548 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda06fc9-230d-42f4-843b-9939e335d00d None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:45.701106 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:45.709190 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94967c3a-d6f9-4c1b-8a85-544ff79ff426 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:45.727909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda06fc9-230d-42f4-843b-9939e335d00d None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:45.749087 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:45.755735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2d475-8500-440e-82f5-85f905838af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:45.756382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2d475-8500-440e-82f5-85f905838af0 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:45.756665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2d475-8500-440e-82f5-85f905838af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:45.756944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2d475-8500-440e-82f5-85f905838af0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:45.771346 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2d475-8500-440e-82f5-85f905838af0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:45.772798 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2d475-8500-440e-82f5-85f905838af0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:45.775718 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ac858-4466-496d-8279-cbb3f930a6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:45.776434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ac858-4466-496d-8279-cbb3f930a6df None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:45.777050 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ac858-4466-496d-8279-cbb3f930a6df None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:45.777423 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ac858-4466-496d-8279-cbb3f930a6df None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:45.778137 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ac858-4466-496d-8279-cbb3f930a6df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:45.778510 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ac858-4466-496d-8279-cbb3f930a6df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:45.780503 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ac858-4466-496d-8279-cbb3f930a6df None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:45.788987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-32b2d475-8500-440e-82f5-85f905838af0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:45.788987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:45.788987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:45.788987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:45.788987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:45.788987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:45.788987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:45.788987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:45.788987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:45.788987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:45.788987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:45.788987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:45.788987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:45.788987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:45.788987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:45.788987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:45.789966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:45.789966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:45.789966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:45.789966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:45.789966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:45.789966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:45.789966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:45.789966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:45.789966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:45.789966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:45.789966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:45.789966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:45.789966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:45.789966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:45.789966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:45.789966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:45.789966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:45.790851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:45.790851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:45.790851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:45.790851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 294/586] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:45.792101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2d475-8500-440e-82f5-85f905838af0 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:45.795665 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 293/587] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:45.796580 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ac858-4466-496d-8279-cbb3f930a6df None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:45.916716 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2d475-8500-440e-82f5-85f905838af0 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:45.921400 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ac858-4466-496d-8279-cbb3f930a6df None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:45.957823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8f6ec-6fa0-47ed-a02c-a4fb47d7f196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:45.958135 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7b940-b6a3-4ea7-b7b2-01731f7f212e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:45.958464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8f6ec-6fa0-47ed-a02c-a4fb47d7f196 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:45.958753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8f6ec-6fa0-47ed-a02c-a4fb47d7f196 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:45.958958 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7b940-b6a3-4ea7-b7b2-01731f7f212e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:45.959058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8f6ec-6fa0-47ed-a02c-a4fb47d7f196 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:45.959318 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7b940-b6a3-4ea7-b7b2-01731f7f212e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:45.959659 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7b940-b6a3-4ea7-b7b2-01731f7f212e None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:45.960311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7b940-b6a3-4ea7-b7b2-01731f7f212e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:45.960612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7b940-b6a3-4ea7-b7b2-01731f7f212e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:45.962020 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7b940-b6a3-4ea7-b7b2-01731f7f212e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:45.973710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8f6ec-6fa0-47ed-a02c-a4fb47d7f196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:45.980739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 294/588] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:45.980739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7b940-b6a3-4ea7-b7b2-01731f7f212e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:45.980948 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8f6ec-6fa0-47ed-a02c-a4fb47d7f196 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:45.998227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-45d8f6ec-6fa0-47ed-a02c-a4fb47d7f196 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:45.998227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:45.998227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:45.998227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:45.998227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:45.998227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:45.998227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:45.998227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:45.998227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:45.998227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:45.998227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:45.998227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:45.998227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:45.998227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:45.998227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:45.998227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:45.999219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:45.999219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:45.999219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:45.999219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:45.999219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:45.999219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:45.999219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:45.999219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:45.999219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:45.999219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:45.999219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:45.999219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:45.999219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:45.999219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:45.999219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:45.999219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:45.999219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:46.001310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:46.001310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:46.001310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:46.001310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 295/589] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:46.001310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8f6ec-6fa0-47ed-a02c-a4fb47d7f196 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:46.002397 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:46.026607 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:46.115464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8f6ec-6fa0-47ed-a02c-a4fb47d7f196 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:46.123939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7b940-b6a3-4ea7-b7b2-01731f7f212e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:46.154807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac365db-541e-437d-b56e-78adbaacbe69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:46.155443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac365db-541e-437d-b56e-78adbaacbe69 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:46.155717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac365db-541e-437d-b56e-78adbaacbe69 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:46.155984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac365db-541e-437d-b56e-78adbaacbe69 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:46.168398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac365db-541e-437d-b56e-78adbaacbe69 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:46.168398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac365db-541e-437d-b56e-78adbaacbe69 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:46.173902 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b7886-457c-4925-8d10-d759d7d5b431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:46.174854 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b7886-457c-4925-8d10-d759d7d5b431 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:46.175361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b7886-457c-4925-8d10-d759d7d5b431 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:46.175875 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b7886-457c-4925-8d10-d759d7d5b431 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:46.182814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-9ac365db-541e-437d-b56e-78adbaacbe69 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 12:09:46.182814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:46.182814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:46.182814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:46.182814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:46.182814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:46.182814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:46.182814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:46.182814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:46.182814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:46.182814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:46.182814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:46.182814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:09:46.182814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:09:46.182814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:09:46.182814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:46.183753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:46.184717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:46.184717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:46.184717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:09:46.184717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:09:46.184717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:09:46.184717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:09:46.184717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:09:46.184717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:09:46.184717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 12:09:46.184717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:46.184717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 296/590] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:46.184717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac365db-541e-437d-b56e-78adbaacbe69 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:46.192306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b7886-457c-4925-8d10-d759d7d5b431 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:46.193792 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b7886-457c-4925-8d10-d759d7d5b431 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:46.206412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-373b7886-457c-4925-8d10-d759d7d5b431 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 28 12:09:46.206412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:46.206412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:46.206412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:46.206412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:46.206412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:46.206412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:46.206412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:46.206412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:46.206412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:46.206412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:46.206412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:46.206412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:09:46.206412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:09:46.206412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:09:46.206412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:46.207394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:46.208957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:46.208957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:46.208957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:09:46.208957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:09:46.208957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:09:46.208957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:09:46.208957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:09:46.208957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:09:46.208957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 28 12:09:46.208957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:46.212239 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 295/591] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:46.213170 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b7886-457c-4925-8d10-d759d7d5b431 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:46.264708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:46.314283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:46.316093 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac365db-541e-437d-b56e-78adbaacbe69 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:46.324397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b7886-457c-4925-8d10-d759d7d5b431 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:46.361170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43c9e3-771b-4ed5-b7a2-da958be0a4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:46.362127 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43c9e3-771b-4ed5-b7a2-da958be0a4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:46.362674 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43c9e3-771b-4ed5-b7a2-da958be0a4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:46.363099 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43c9e3-771b-4ed5-b7a2-da958be0a4c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:46.363944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43c9e3-771b-4ed5-b7a2-da958be0a4c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:46.364396 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43c9e3-771b-4ed5-b7a2-da958be0a4c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:46.365458 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381dccb5-6117-4d9b-81e5-1c4f4e5ee91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:46.366180 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dccb5-6117-4d9b-81e5-1c4f4e5ee91e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:46.366396 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43c9e3-771b-4ed5-b7a2-da958be0a4c7 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:46.366637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dccb5-6117-4d9b-81e5-1c4f4e5ee91e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:46.367003 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dccb5-6117-4d9b-81e5-1c4f4e5ee91e None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:46.367868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381dccb5-6117-4d9b-81e5-1c4f4e5ee91e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:46.368399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381dccb5-6117-4d9b-81e5-1c4f4e5ee91e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:46.370044 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381dccb5-6117-4d9b-81e5-1c4f4e5ee91e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:46.382839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 297/592] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:46.383964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43c9e3-771b-4ed5-b7a2-da958be0a4c7 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:46.400162 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 296/593] 10.222.0.60 () {56 vars in 1139 bytes} [Wed Sep 28 12:09:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:46.400162 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381dccb5-6117-4d9b-81e5-1c4f4e5ee91e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:46.504210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43c9e3-771b-4ed5-b7a2-da958be0a4c7 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:46.507121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381dccb5-6117-4d9b-81e5-1c4f4e5ee91e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:46.527679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:46.544619 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dab0d-1ec0-4efc-8c91-632d944b4fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:46.545358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dab0d-1ec0-4efc-8c91-632d944b4fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:46.545705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dab0d-1ec0-4efc-8c91-632d944b4fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:46.546117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dab0d-1ec0-4efc-8c91-632d944b4fa0 None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:46.546996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834dab0d-1ec0-4efc-8c91-632d944b4fa0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:46.547467 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834dab0d-1ec0-4efc-8c91-632d944b4fa0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:46.547977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf0b00-b46c-43e5-8eaf-5c96f54b41a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:46.548801 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf0b00-b46c-43e5-8eaf-5c96f54b41a5 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:46.549221 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834dab0d-1ec0-4efc-8c91-632d944b4fa0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:46.549598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf0b00-b46c-43e5-8eaf-5c96f54b41a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:46.550049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf0b00-b46c-43e5-8eaf-5c96f54b41a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:46.557615 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf0b00-b46c-43e5-8eaf-5c96f54b41a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:46.559583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf0b00-b46c-43e5-8eaf-5c96f54b41a5 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:46.572357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-e1bf0b00-b46c-43e5-8eaf-5c96f54b41a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:46.572357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:46.572357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:46.572357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:46.572357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:46.572357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:46.572357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:46.572357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:46.572357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:46.572357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:46.572357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:46.572357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:46.572357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:09:46.572357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:09:46.572357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:09:46.574014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:09:46.574014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:46.574014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:46.574014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:46.574014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:46.574014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:46.574014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:46.574014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:46.574014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:46.574014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:46.574014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:46.574014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:46.574014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:46.574014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:46.574014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:46.574014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:46.574014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:46.574904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:09:46.574904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:09:46.574904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:09:46.574904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:09:46.574904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:09:46.574904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:09:46.574904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:46.574904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:46.575363 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 298/594] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Sep 28 12:09:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:46.575363 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dab0d-1ec0-4efc-8c91-632d944b4fa0 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:46.577121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 297/595] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:46.578304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf0b00-b46c-43e5-8eaf-5c96f54b41a5 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:46.598640 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:46.693858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf0b00-b46c-43e5-8eaf-5c96f54b41a5 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:46.698863 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834dab0d-1ec0-4efc-8c91-632d944b4fa0 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:46.729724 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd891324-edf9-4203-8644-75c38bd8ebe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:46.730359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd891324-edf9-4203-8644-75c38bd8ebe3 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:46.730639 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd891324-edf9-4203-8644-75c38bd8ebe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:46.730907 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd891324-edf9-4203-8644-75c38bd8ebe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:46.738216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43073e1-c6fc-4aaf-87fc-3cfbd9f67787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:46.738919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43073e1-c6fc-4aaf-87fc-3cfbd9f67787 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:46.739261 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43073e1-c6fc-4aaf-87fc-3cfbd9f67787 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:46.739597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43073e1-c6fc-4aaf-87fc-3cfbd9f67787 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:46.742473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd891324-edf9-4203-8644-75c38bd8ebe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:46.743937 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd891324-edf9-4203-8644-75c38bd8ebe3 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:46.750689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43073e1-c6fc-4aaf-87fc-3cfbd9f67787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:46.752635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43073e1-c6fc-4aaf-87fc-3cfbd9f67787 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:46.759331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-cd891324-edf9-4203-8644-75c38bd8ebe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:46.759331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:46.759331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:46.759331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:46.759331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:46.759331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:46.759331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:46.759331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:46.759331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:46.759331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:46.759331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:46.759331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:46.759331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:46.759331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:46.759331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:46.759331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:46.760260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:46.760260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:46.760260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:46.760260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:46.760260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:46.760260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:46.760260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:46.760260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:46.760260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:46.760260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:46.760260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:46.760260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:46.760260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:46.760260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:46.760260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:46.760260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:46.760260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:46.761704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:46.761704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:46.761704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:46.765062 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 298/596] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:46.765917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd891324-edf9-4203-8644-75c38bd8ebe3 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:46.769678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-b43073e1-c6fc-4aaf-87fc-3cfbd9f67787 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:46.769678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:46.769678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:46.769678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:46.769678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:46.769678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:46.769678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:46.769678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:46.769678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:46.769678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:46.769678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:46.769678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:46.769678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:46.769678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:46.769678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:46.769678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:46.771531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:46.771531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:46.771531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:46.771531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:46.771531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:46.771531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:46.771531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:46.771531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:46.771531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:46.771531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:46.771531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:46.771531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:46.771531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:46.771531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:46.771531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:46.771531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:46.771531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:46.773974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:46.773974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:46.773974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:46.773974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 299/597] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:46.773974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43073e1-c6fc-4aaf-87fc-3cfbd9f67787 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:46.807678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:46.831237 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:46.879183 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd891324-edf9-4203-8644-75c38bd8ebe3 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:46.894365 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43073e1-c6fc-4aaf-87fc-3cfbd9f67787 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:46.930584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943df553-3b82-4abb-a722-160c285c586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:46.931657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943df553-3b82-4abb-a722-160c285c586c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:46.932201 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943df553-3b82-4abb-a722-160c285c586c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:46.932619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943df553-3b82-4abb-a722-160c285c586c None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:46.933639 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943df553-3b82-4abb-a722-160c285c586c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:46.933898 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817c5fe-6852-4674-82e7-cff81da074c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:46.934317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943df553-3b82-4abb-a722-160c285c586c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:46.934517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817c5fe-6852-4674-82e7-cff81da074c6 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:46.934798 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817c5fe-6852-4674-82e7-cff81da074c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:46.935067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817c5fe-6852-4674-82e7-cff81da074c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:46.935725 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817c5fe-6852-4674-82e7-cff81da074c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:46.936027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817c5fe-6852-4674-82e7-cff81da074c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:46.936793 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943df553-3b82-4abb-a722-160c285c586c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:46.937467 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817c5fe-6852-4674-82e7-cff81da074c6 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:46.961175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 299/598] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:09:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:46.961175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943df553-3b82-4abb-a722-160c285c586c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:46.961481 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 300/599] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:46.965256 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817c5fe-6852-4674-82e7-cff81da074c6 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:47.071246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943df553-3b82-4abb-a722-160c285c586c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:47.075638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:47.088018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817c5fe-6852-4674-82e7-cff81da074c6 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:47.097587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:47.111165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe01496-2693-410b-b3a0-032214960f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:47.111779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe01496-2693-410b-b3a0-032214960f5e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:47.112069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe01496-2693-410b-b3a0-032214960f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:47.112324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe01496-2693-410b-b3a0-032214960f5e None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:47.112969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe01496-2693-410b-b3a0-032214960f5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:47.113314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe01496-2693-410b-b3a0-032214960f5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:47.114667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe01496-2693-410b-b3a0-032214960f5e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:47.125181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02531691-f749-4cd7-b909-6146906b48cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:47.126061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02531691-f749-4cd7-b909-6146906b48cf None admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:47.126573 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02531691-f749-4cd7-b909-6146906b48cf None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:47.126922 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 300/600] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:47.127174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02531691-f749-4cd7-b909-6146906b48cf None admin] PATH_INFO: `/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/2354a4fd72cc42c58d6830eecb1e6253/roles/60963384f3e54d9cbfbc8d12023a6cd9` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:47.127758 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe01496-2693-410b-b3a0-032214960f5e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:47.147276 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02531691-f749-4cd7-b909-6146906b48cf None admin] RBAC: Authorizing `identity:create_grant(project_id=6335d0db1f4844c39c255909189e0ce9, user_id=2354a4fd72cc42c58d6830eecb1e6253, role_id=60963384f3e54d9cbfbc8d12023a6cd9)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:47.148921 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02531691-f749-4cd7-b909-6146906b48cf None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:47.168598 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 301/601] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:09:46 2022] PUT /identity/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/2354a4fd72cc42c58d6830eecb1e6253/roles/60963384f3e54d9cbfbc8d12023a6cd9 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:09:47.168598 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd3c5e-8ce0-44e1-ad7f-53b4c94d469c None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:47.168598 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd3c5e-8ce0-44e1-ad7f-53b4c94d469c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:47.168598 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd3c5e-8ce0-44e1-ad7f-53b4c94d469c None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:47.173114 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 302/602] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:47.174234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd3c5e-8ce0-44e1-ad7f-53b4c94d469c None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:47.268358 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe01496-2693-410b-b3a0-032214960f5e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:47.307327 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb386e-d8dd-4527-93f7-ec21b8481325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:47.307976 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb386e-d8dd-4527-93f7-ec21b8481325 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:47.308259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb386e-d8dd-4527-93f7-ec21b8481325 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:47.308532 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb386e-d8dd-4527-93f7-ec21b8481325 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:47.321383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb386e-d8dd-4527-93f7-ec21b8481325 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:47.324328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb386e-d8dd-4527-93f7-ec21b8481325 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:47.336459 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd3c5e-8ce0-44e1-ad7f-53b4c94d469c None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:47.340553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-39cb386e-d8dd-4527-93f7-ec21b8481325 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:47.340553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:47.340553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:47.340553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:47.340553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:47.340553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:47.340553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:47.340553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:47.340553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:47.340553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:47.340553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:47.340553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:47.340553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:09:47.340553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:09:47.340553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:09:47.341445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:09:47.341445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:47.341445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:47.341445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:47.341445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:47.341445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:47.341445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:47.341445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:47.341445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:47.341445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:47.341445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:47.341445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:47.341445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:47.341445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:47.341445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:47.341445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:47.341445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:47.342337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:09:47.342337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:09:47.342337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:09:47.342337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:09:47.342337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:09:47.342337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:09:47.342337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:47.342337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:47.345922 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 301/603] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:47.347382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362e314-f731-41a4-8df5-c7a5b61f5088 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:47.347877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362e314-f731-41a4-8df5-c7a5b61f5088 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:47.348378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362e314-f731-41a4-8df5-c7a5b61f5088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:47.368723 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e362e314-f731-41a4-8df5-c7a5b61f5088 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:47.370121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e362e314-f731-41a4-8df5-c7a5b61f5088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:47.377774 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2310c-4d3d-4987-bb9b-1462092142ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:47.378413 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2310c-4d3d-4987-bb9b-1462092142ee None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:47.378691 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2310c-4d3d-4987-bb9b-1462092142ee None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:47.378963 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2310c-4d3d-4987-bb9b-1462092142ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:47.386776 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2310c-4d3d-4987-bb9b-1462092142ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:47.389239 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2310c-4d3d-4987-bb9b-1462092142ee None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:47.392240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:47.404170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-bdb2310c-4d3d-4987-bb9b-1462092142ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:47.404170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:47.404170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:47.404170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:47.404170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:47.404170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:47.404170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:47.404170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:47.404170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:47.404170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:47.404170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:47.404170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:47.404170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:09:47.404170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:09:47.404170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:09:47.405094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:09:47.405094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:47.405094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:47.405094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:47.405094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:47.405094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:47.405094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:47.405094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:47.405094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:47.405094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:47.405094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:47.405094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:47.405094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:47.405094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:47.405094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:47.405094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:47.405094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:47.406036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:09:47.406036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:09:47.406036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:09:47.406036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:09:47.406036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:09:47.406036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:09:47.406036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:47.406036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:47.406036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 303/604] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:47.406510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2310c-4d3d-4987-bb9b-1462092142ee None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:47.430057 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:47.470433 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 302/605] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:47.471533 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362e314-f731-41a4-8df5-c7a5b61f5088 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:47.524765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2310c-4d3d-4987-bb9b-1462092142ee None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:47.563897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b9640-1559-45ed-b956-48421598ac26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:47.564867 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b9640-1559-45ed-b956-48421598ac26 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:47.565534 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b9640-1559-45ed-b956-48421598ac26 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:47.566023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b9640-1559-45ed-b956-48421598ac26 None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:47.567162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b9640-1559-45ed-b956-48421598ac26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:47.567666 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b9640-1559-45ed-b956-48421598ac26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:47.569297 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b9640-1559-45ed-b956-48421598ac26 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:47.583800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 304/606] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:09:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:47.590355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b9640-1559-45ed-b956-48421598ac26 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:47.673535 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362e314-f731-41a4-8df5-c7a5b61f5088 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:47.715953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac191dfd-a71c-4421-91e2-57a202ae148d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:47.716913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac191dfd-a71c-4421-91e2-57a202ae148d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:47.717453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac191dfd-a71c-4421-91e2-57a202ae148d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:47.717955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac191dfd-a71c-4421-91e2-57a202ae148d None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:47.718839 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac191dfd-a71c-4421-91e2-57a202ae148d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:47.719970 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac191dfd-a71c-4421-91e2-57a202ae148d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:47.721811 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:47.723564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac191dfd-a71c-4421-91e2-57a202ae148d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:47.739252 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 303/607] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:09:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:47.741432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f4450-ccef-49ce-8eed-7a1728c36fa1 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:47.741735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f4450-ccef-49ce-8eed-7a1728c36fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:47.742011 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f4450-ccef-49ce-8eed-7a1728c36fa1 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:47.745201 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 304/608] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:47.746149 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f4450-ccef-49ce-8eed-7a1728c36fa1 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:47.783268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b9640-1559-45ed-b956-48421598ac26 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:47.790856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:47.823330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0dba57-38e2-40cb-ab9c-5419c8a182b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:47.823981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0dba57-38e2-40cb-ab9c-5419c8a182b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:47.824301 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0dba57-38e2-40cb-ab9c-5419c8a182b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:47.824529 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0dba57-38e2-40cb-ab9c-5419c8a182b0 None admin] PATH_INFO: `/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/8e08e00bcb5f47ffab58974ba376938a/roles/60963384f3e54d9cbfbc8d12023a6cd9` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:47.840989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0dba57-38e2-40cb-ab9c-5419c8a182b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6335d0db1f4844c39c255909189e0ce9, user_id=8e08e00bcb5f47ffab58974ba376938a, role_id=60963384f3e54d9cbfbc8d12023a6cd9)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:47.842402 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0dba57-38e2-40cb-ab9c-5419c8a182b0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:47.856926 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 305/609] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:09:47 2022] PUT /identity/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/8e08e00bcb5f47ffab58974ba376938a/roles/60963384f3e54d9cbfbc8d12023a6cd9 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:09:47.858393 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcf553-6515-4532-9049-4addad5e31f3 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:47.858506 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcf553-6515-4532-9049-4addad5e31f3 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:47.858713 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dcf553-6515-4532-9049-4addad5e31f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:47.873320 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f4450-ccef-49ce-8eed-7a1728c36fa1 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:47.881482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-61dcf553-6515-4532-9049-4addad5e31f3 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:47.883179 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-61dcf553-6515-4532-9049-4addad5e31f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:47.917387 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82464f03-7bae-4b02-b82a-2f4dcb913ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:47.918549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82464f03-7bae-4b02-b82a-2f4dcb913ef1 None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:47.919052 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82464f03-7bae-4b02-b82a-2f4dcb913ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:47.919705 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82464f03-7bae-4b02-b82a-2f4dcb913ef1 None admin] PATH_INFO: `/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/ba75586d3a7d4278ad6c0c31a1bed3df/roles/60963384f3e54d9cbfbc8d12023a6cd9` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:47.943678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82464f03-7bae-4b02-b82a-2f4dcb913ef1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6335d0db1f4844c39c255909189e0ce9, user_id=ba75586d3a7d4278ad6c0c31a1bed3df, role_id=60963384f3e54d9cbfbc8d12023a6cd9)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:47.945686 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82464f03-7bae-4b02-b82a-2f4dcb913ef1 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:47.962717 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 305/610] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:09:47 2022] PUT /identity/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/ba75586d3a7d4278ad6c0c31a1bed3df/roles/60963384f3e54d9cbfbc8d12023a6cd9 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:09:47.964081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82464f03-7bae-4b02-b82a-2f4dcb913ef1 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:47.983580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 306/611] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:48.078982 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:48.101552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82464f03-7bae-4b02-b82a-2f4dcb913ef1 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:48.142129 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6945fcf-b58e-45ae-838b-a8d0edde9922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:48.143161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6945fcf-b58e-45ae-838b-a8d0edde9922 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:48.143664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6945fcf-b58e-45ae-838b-a8d0edde9922 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:48.144195 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6945fcf-b58e-45ae-838b-a8d0edde9922 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:48.157195 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6945fcf-b58e-45ae-838b-a8d0edde9922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:48.163924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-a6945fcf-b58e-45ae-838b-a8d0edde9922 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:48.163924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:48.163924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:48.163924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:48.163924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:48.163924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:48.163924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:48.163924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:48.163924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:48.163924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:48.163924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:48.163924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:48.163924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:48.163924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:48.163924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:48.163924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:48.164965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:48.164965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:48.164965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:48.164965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:48.164965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:48.164965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:48.164965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:48.164965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:48.164965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:48.164965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:48.164965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:48.164965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:48.164965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:48.164965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:48.164965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:48.164965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:48.165853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:48.165853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:48.165853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:48.165853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:48.165853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:48.165853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:48.165853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:48.165853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:48.165853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:48.165853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:48.165853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:48.165853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:48.168722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 306/612] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:48.176006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dcf553-6515-4532-9049-4addad5e31f3 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:48.221705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:48.285177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dcf553-6515-4532-9049-4addad5e31f3 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:48.331340 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6945fcf-b58e-45ae-838b-a8d0edde9922 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:48.338901 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95090692-9813-4bda-8f97-3b033b974572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:48.339525 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95090692-9813-4bda-8f97-3b033b974572 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:48.339802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95090692-9813-4bda-8f97-3b033b974572 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:48.340070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95090692-9813-4bda-8f97-3b033b974572 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:48.340727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95090692-9813-4bda-8f97-3b033b974572 None admin] RBAC: Adding query filter params (name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:48.341025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95090692-9813-4bda-8f97-3b033b974572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:48.342515 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95090692-9813-4bda-8f97-3b033b974572 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:48.359914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 307/613] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:09:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:48.369159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95090692-9813-4bda-8f97-3b033b974572 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:48.523251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6945fcf-b58e-45ae-838b-a8d0edde9922 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:48.552685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95090692-9813-4bda-8f97-3b033b974572 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:48.554632 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:48.597950 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf56588-3d67-4779-b8eb-bacb0395b1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:48.598595 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf56588-3d67-4779-b8eb-bacb0395b1d1 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:48.598876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf56588-3d67-4779-b8eb-bacb0395b1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:48.599148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf56588-3d67-4779-b8eb-bacb0395b1d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:48.603852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4aa5f3-303f-478d-b662-9dfacc1e8685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:48.604758 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4aa5f3-303f-478d-b662-9dfacc1e8685 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:48.605375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4aa5f3-303f-478d-b662-9dfacc1e8685 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:48.605854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4aa5f3-303f-478d-b662-9dfacc1e8685 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:48.618548 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4aa5f3-303f-478d-b662-9dfacc1e8685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:48.620481 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4aa5f3-303f-478d-b662-9dfacc1e8685 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:48.626568 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf56588-3d67-4779-b8eb-bacb0395b1d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:48.631591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-acf56588-3d67-4779-b8eb-bacb0395b1d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:48.631591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:48.631591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:48.631591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:48.631591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:48.631591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:48.631591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:48.631591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:48.631591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:48.631591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:48.631591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:48.631591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:48.631591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:48.631591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:48.631591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:48.631591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:48.632523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:48.632523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:48.632523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:48.632523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:48.632523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:48.632523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:48.632523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:48.632523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:48.632523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:48.632523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:48.632523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:48.632523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:48.632523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:48.632523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:48.632523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:48.632523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:48.633376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:48.633376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:48.633376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:48.633376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:48.633376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:48.633376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:48.633376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:48.633376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:48.633376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:48.633376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:48.633376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:48.633376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:48.641212 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 307/614] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:48.642250 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf56588-3d67-4779-b8eb-bacb0395b1d1 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:48.647450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-ef4aa5f3-303f-478d-b662-9dfacc1e8685 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:48.647450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:48.647450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:48.647450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:48.647450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:48.647450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:48.647450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:48.647450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:48.647450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:48.647450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:48.647450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:48.647450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:48.647450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:48.647450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:48.647450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:48.647450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:48.648371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:48.648371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:48.648371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:48.648371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:48.648371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:48.648371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:48.648371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:48.648371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:48.648371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:48.648371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:48.648371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:48.648371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:48.648371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:48.648371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:48.648371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:48.648371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:48.648371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:48.649298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:48.649298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:48.649298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:48.650147 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:48.652589 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 308/615] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:48.660292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4aa5f3-303f-478d-b662-9dfacc1e8685 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:48.832883 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf56588-3d67-4779-b8eb-bacb0395b1d1 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:48.853772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4aa5f3-303f-478d-b662-9dfacc1e8685 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:48.884333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95168ea-0874-4a2f-8f1f-aaaf9671ed10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:48.884862 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95168ea-0874-4a2f-8f1f-aaaf9671ed10 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:48.885184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95168ea-0874-4a2f-8f1f-aaaf9671ed10 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:48.885592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95168ea-0874-4a2f-8f1f-aaaf9671ed10 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:48.886271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95168ea-0874-4a2f-8f1f-aaaf9671ed10 None admin] RBAC: Adding query filter params (name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:48.886575 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95168ea-0874-4a2f-8f1f-aaaf9671ed10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:48.888232 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95168ea-0874-4a2f-8f1f-aaaf9671ed10 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:48.897475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 308/616] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:09:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:48.903779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95168ea-0874-4a2f-8f1f-aaaf9671ed10 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:48.936456 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862b423-b198-4188-8d00-9e56a1df9493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:48.937069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862b423-b198-4188-8d00-9e56a1df9493 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:48.937375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862b423-b198-4188-8d00-9e56a1df9493 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:48.937654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862b423-b198-4188-8d00-9e56a1df9493 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:48.938338 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862b423-b198-4188-8d00-9e56a1df9493 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:48.938641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862b423-b198-4188-8d00-9e56a1df9493 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:48.940026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862b423-b198-4188-8d00-9e56a1df9493 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:48.952874 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 309/617] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:48.958262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862b423-b198-4188-8d00-9e56a1df9493 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:48.978457 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:49.035469 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:49.117274 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95168ea-0874-4a2f-8f1f-aaaf9671ed10 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:49.151311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862b423-b198-4188-8d00-9e56a1df9493 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:49.208851 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025a3e4-db20-4c26-8c5c-d74fc89dfd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:49.209774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025a3e4-db20-4c26-8c5c-d74fc89dfd0d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:49.210211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025a3e4-db20-4c26-8c5c-d74fc89dfd0d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:49.210586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025a3e4-db20-4c26-8c5c-d74fc89dfd0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:49.211141 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31ecd1-6fec-4c6a-8afc-6c4d26f15cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:49.211936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31ecd1-6fec-4c6a-8afc-6c4d26f15cdd None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:49.212360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31ecd1-6fec-4c6a-8afc-6c4d26f15cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:49.213031 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31ecd1-6fec-4c6a-8afc-6c4d26f15cdd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:49.221493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31ecd1-6fec-4c6a-8afc-6c4d26f15cdd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:49.223163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d31ecd1-6fec-4c6a-8afc-6c4d26f15cdd None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:49.237749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-7d31ecd1-6fec-4c6a-8afc-6c4d26f15cdd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 12:09:49.237749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:49.237749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:49.237749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:49.237749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:49.237749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:49.237749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:49.237749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:49.237749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:49.237749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:49.237749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:49.237749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:49.237749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:09:49.237749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:09:49.237749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:09:49.237749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:49.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:49.240176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:49.240176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:49.240176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:09:49.240176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:09:49.240176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:09:49.240176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:09:49.240176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:09:49.240176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:09:49.240176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 12:09:49.240176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:49.240923 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025a3e4-db20-4c26-8c5c-d74fc89dfd0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:49.242479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025a3e4-db20-4c26-8c5c-d74fc89dfd0d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:49.242650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 310/618] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:09:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:49.248145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31ecd1-6fec-4c6a-8afc-6c4d26f15cdd None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:49.268160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-d025a3e4-db20-4c26-8c5c-d74fc89dfd0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:49.268160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:49.268160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:49.268160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:49.268160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:49.268160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:49.268160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:49.268160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:49.268160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:49.268160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:49.268160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:49.268160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:49.268160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:49.268160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:49.268160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:49.268160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:49.269543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:49.269543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:49.269543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:49.269543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:49.269543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:49.269543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:49.269543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:49.269543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:49.269543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:49.269543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:49.269543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:49.269543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:49.269543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:49.269543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:49.269543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:49.269543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:49.269543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:49.270601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:49.270601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:49.270601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:49.278443 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 309/619] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:49.284944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025a3e4-db20-4c26-8c5c-d74fc89dfd0d None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:49.353495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:49.377806 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31ecd1-6fec-4c6a-8afc-6c4d26f15cdd None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:49.429799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b3a71-4458-4f08-b28b-f8abdf69e01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:49.430418 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b3a71-4458-4f08-b28b-f8abdf69e01b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:49.430701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b3a71-4458-4f08-b28b-f8abdf69e01b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:49.431117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b3a71-4458-4f08-b28b-f8abdf69e01b None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:49.431944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b3a71-4458-4f08-b28b-f8abdf69e01b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:49.432254 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b3a71-4458-4f08-b28b-f8abdf69e01b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:49.433726 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b3a71-4458-4f08-b28b-f8abdf69e01b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:49.452894 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 311/620] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Sep 28 12:09:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:49.458383 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b3a71-4458-4f08-b28b-f8abdf69e01b None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:49.461048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025a3e4-db20-4c26-8c5c-d74fc89dfd0d None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:49.495349 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:49.520101 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9234c-ae36-408a-b021-b524e34f3b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:49.520862 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9234c-ae36-408a-b021-b524e34f3b84 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:49.521158 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9234c-ae36-408a-b021-b524e34f3b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:49.521467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9234c-ae36-408a-b021-b524e34f3b84 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:49.522193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9234c-ae36-408a-b021-b524e34f3b84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:49.522502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9234c-ae36-408a-b021-b524e34f3b84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:49.523882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9234c-ae36-408a-b021-b524e34f3b84 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:49.543025 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 310/621] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:49.545202 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9234c-ae36-408a-b021-b524e34f3b84 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:49.645447 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b3a71-4458-4f08-b28b-f8abdf69e01b None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:49.678523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9234c-ae36-408a-b021-b524e34f3b84 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:49.680109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:49.681748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8399c2-d7b9-4895-bc13-9390d2e614dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:49.682398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8399c2-d7b9-4895-bc13-9390d2e614dc None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:49.682686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8399c2-d7b9-4895-bc13-9390d2e614dc None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:49.683064 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8399c2-d7b9-4895-bc13-9390d2e614dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:49.696038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8399c2-d7b9-4895-bc13-9390d2e614dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:49.697892 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8399c2-d7b9-4895-bc13-9390d2e614dc None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:49.716675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-1c8399c2-d7b9-4895-bc13-9390d2e614dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:49.716675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:49.716675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:49.716675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:49.716675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:49.716675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:49.716675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:49.716675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:49.716675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:49.716675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:49.716675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:49.716675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:49.716675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:49.716675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:49.716675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:49.716675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:49.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:49.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:49.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:49.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:49.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:49.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:49.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:49.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:49.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:49.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:49.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:49.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:49.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:49.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:49.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:49.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:49.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:49.719270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:49.719270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:49.719270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:49.719270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 312/622] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:49.719724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc438f-f24b-4134-b562-c25a33cde480 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:49.719724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc438f-f24b-4134-b562-c25a33cde480 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:49.720155 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc438f-f24b-4134-b562-c25a33cde480 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:49.721490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 313/623] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:49.723589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4534cd-f458-4120-9078-7050ae417d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:49.724258 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4534cd-f458-4120-9078-7050ae417d45 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:49.724534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4534cd-f458-4120-9078-7050ae417d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:49.724804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4534cd-f458-4120-9078-7050ae417d45 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:49.726799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f200504-1cd5-4661-8ac2-bd546b319a83 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:49.727424 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f200504-1cd5-4661-8ac2-bd546b319a83 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:49.727891 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f200504-1cd5-4661-8ac2-bd546b319a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:49.732840 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4534cd-f458-4120-9078-7050ae417d45 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:49.734389 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4534cd-f458-4120-9078-7050ae417d45 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:49.748163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-2b4534cd-f458-4120-9078-7050ae417d45 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:09:49.748163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:49.748163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:49.748163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:49.748163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:49.748163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:49.748163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:49.748163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:49.748163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:49.748163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:49.748163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:49.748163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:49.748163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:09:49.748163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:09:49.748163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:09:49.748163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:49.749121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:49.750283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:49.750283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:49.750283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:09:49.750283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:09:49.750283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:09:49.750283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:09:49.750283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:09:49.750283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:09:49.750283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:09:49.750283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:49.752016 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-6f200504-1cd5-4661-8ac2-bd546b319a83 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:49.753445 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-6f200504-1cd5-4661-8ac2-bd546b319a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:49.754661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 311/624] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 12:09:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:49.755481 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4534cd-f458-4120-9078-7050ae417d45 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:49.802002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:49.839472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 314/625] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:49.840735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f200504-1cd5-4661-8ac2-bd546b319a83 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:49.904130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4534cd-f458-4120-9078-7050ae417d45 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:49.969797 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f283222-3532-41cd-b00d-9ef14bb8e9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:49.970895 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f283222-3532-41cd-b00d-9ef14bb8e9ca None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:49.972350 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f283222-3532-41cd-b00d-9ef14bb8e9ca None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:49.972983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f283222-3532-41cd-b00d-9ef14bb8e9ca None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:49.974901 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f283222-3532-41cd-b00d-9ef14bb8e9ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:49.975590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f283222-3532-41cd-b00d-9ef14bb8e9ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:49.977794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f283222-3532-41cd-b00d-9ef14bb8e9ca None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:50.002605 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 312/626] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:50.007595 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f283222-3532-41cd-b00d-9ef14bb8e9ca None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:50.051563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f200504-1cd5-4661-8ac2-bd546b319a83 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:50.088062 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:50.141508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37198c3-767a-4753-8e2e-2967e7c3e926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:50.142846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37198c3-767a-4753-8e2e-2967e7c3e926 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:50.143357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37198c3-767a-4753-8e2e-2967e7c3e926 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:50.144137 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37198c3-767a-4753-8e2e-2967e7c3e926 None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:50.145366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37198c3-767a-4753-8e2e-2967e7c3e926 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:50.145876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37198c3-767a-4753-8e2e-2967e7c3e926 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:50.146761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:50.147846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37198c3-767a-4753-8e2e-2967e7c3e926 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:50.154524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f283222-3532-41cd-b00d-9ef14bb8e9ca None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:50.168257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 315/627] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 12:09:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:50.169444 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37198c3-767a-4753-8e2e-2967e7c3e926 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:50.199901 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b37286-952e-4688-92c1-2548e89316be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:50.200594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b37286-952e-4688-92c1-2548e89316be None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:50.200877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b37286-952e-4688-92c1-2548e89316be None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:50.201321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b37286-952e-4688-92c1-2548e89316be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:50.211175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b37286-952e-4688-92c1-2548e89316be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:50.212844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b37286-952e-4688-92c1-2548e89316be None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:50.228031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-d5b37286-952e-4688-92c1-2548e89316be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:50.228031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:50.228031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:50.228031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:50.228031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:50.228031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:50.228031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:50.228031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:50.228031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:50.228031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:50.228031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:50.228031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:50.228031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:09:50.228031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:09:50.228031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:09:50.228977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:09:50.228977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:50.228977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:50.228977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:50.228977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:50.228977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:50.228977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:50.228977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:50.228977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:50.228977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:50.228977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:50.228977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:50.228977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:50.228977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:50.228977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:50.228977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:50.228977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:50.230005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:09:50.230005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:09:50.230005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:09:50.230005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:09:50.230005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:09:50.230005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:09:50.230005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:50.230005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:50.230005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 313/628] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:50.230884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b37286-952e-4688-92c1-2548e89316be None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:50.296456 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37198c3-767a-4753-8e2e-2967e7c3e926 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:50.331599 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35950d39-072c-4036-9901-1a4aa14bc424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:50.332242 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35950d39-072c-4036-9901-1a4aa14bc424 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:50.332519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35950d39-072c-4036-9901-1a4aa14bc424 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:50.332789 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35950d39-072c-4036-9901-1a4aa14bc424 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:50.343276 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35950d39-072c-4036-9901-1a4aa14bc424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:50.348640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-35950d39-072c-4036-9901-1a4aa14bc424 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:50.348640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:50.348640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:50.348640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:50.348640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:50.348640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:50.348640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:50.348640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:50.348640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:50.348640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:50.348640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:50.348640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:50.348640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:50.348640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:50.348640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:50.348640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:50.350837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:50.350837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:50.350837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:50.350837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:50.350837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:50.350837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:50.350837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:50.350837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:50.350837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:50.350837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:50.350837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:50.350837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:50.350837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:50.350837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:50.350837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:50.350837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:50.352825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:50.352825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:50.352825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:50.352825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:50.352825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:50.352825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:50.352825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:50.352825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:50.352825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:50.352825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:50.352825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:50.352825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:50.352825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 316/629] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:50.352825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35950d39-072c-4036-9901-1a4aa14bc424 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:50.361155 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b37286-952e-4688-92c1-2548e89316be None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:50.383556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:50.401667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d3684-0a04-469c-8e28-d16ecd8ab0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:50.402827 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d3684-0a04-469c-8e28-d16ecd8ab0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:50.403333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d3684-0a04-469c-8e28-d16ecd8ab0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:50.403757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d3684-0a04-469c-8e28-d16ecd8ab0b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:50.436514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:50.461882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35950d39-072c-4036-9901-1a4aa14bc424 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:50.488848 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d3684-0a04-469c-8e28-d16ecd8ab0b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:50.490686 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d3684-0a04-469c-8e28-d16ecd8ab0b8 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:50.497820 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28965a1-847c-4cac-a4c9-b1ac38539f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:50.498426 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28965a1-847c-4cac-a4c9-b1ac38539f67 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:50.498703 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28965a1-847c-4cac-a4c9-b1ac38539f67 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:50.498974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28965a1-847c-4cac-a4c9-b1ac38539f67 None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:50.499658 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28965a1-847c-4cac-a4c9-b1ac38539f67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:50.499961 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28965a1-847c-4cac-a4c9-b1ac38539f67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:50.502035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28965a1-847c-4cac-a4c9-b1ac38539f67 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:50.514217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 317/630] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:09:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:50.515046 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28965a1-847c-4cac-a4c9-b1ac38539f67 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:50.596155 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 314/631] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:50.597425 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81fbf7-e841-4954-9050-64c0ae3365de None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:50.598024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81fbf7-e841-4954-9050-64c0ae3365de None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:50.598024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81fbf7-e841-4954-9050-64c0ae3365de None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:50.599159 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 315/632] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:50.601280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81fbf7-e841-4954-9050-64c0ae3365de None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:50.609719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:50.619324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28965a1-847c-4cac-a4c9-b1ac38539f67 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:50.653470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7fa59-c656-4079-b0ea-bfbc6357f83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:50.654079 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7fa59-c656-4079-b0ea-bfbc6357f83b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:50.654362 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7fa59-c656-4079-b0ea-bfbc6357f83b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:50.654637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7fa59-c656-4079-b0ea-bfbc6357f83b None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:50.655293 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7fa59-c656-4079-b0ea-bfbc6357f83b None admin] RBAC: Adding query filter params (name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:50.655598 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7fa59-c656-4079-b0ea-bfbc6357f83b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:50.656467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:50.657656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7fa59-c656-4079-b0ea-bfbc6357f83b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:50.670328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 318/633] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:09:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:50.670328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc485a06-f190-4b99-8ef5-6e8aec371bf5 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:50.670328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc485a06-f190-4b99-8ef5-6e8aec371bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:50.670328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc485a06-f190-4b99-8ef5-6e8aec371bf5 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:50.670328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 319/634] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:50.670328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc485a06-f190-4b99-8ef5-6e8aec371bf5 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:50.704338 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81fbf7-e841-4954-9050-64c0ae3365de None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:50.738575 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572143e9-0f85-4752-a3d0-2f93446f97df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:50.739273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572143e9-0f85-4752-a3d0-2f93446f97df None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:50.739503 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572143e9-0f85-4752-a3d0-2f93446f97df None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:50.739778 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572143e9-0f85-4752-a3d0-2f93446f97df None admin] PATH_INFO: `/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/975a8d7349994520bcbfb5f75e52aac7/roles/60963384f3e54d9cbfbc8d12023a6cd9` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:50.753736 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572143e9-0f85-4752-a3d0-2f93446f97df None admin] RBAC: Authorizing `identity:create_grant(project_id=6335d0db1f4844c39c255909189e0ce9, user_id=975a8d7349994520bcbfb5f75e52aac7, role_id=60963384f3e54d9cbfbc8d12023a6cd9)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:50.755169 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572143e9-0f85-4752-a3d0-2f93446f97df None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:50.768695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 316/635] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:09:50 2022] PUT /identity/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/975a8d7349994520bcbfb5f75e52aac7/roles/60963384f3e54d9cbfbc8d12023a6cd9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:09:50.770105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc6d8a-752d-456b-9c95-a5c8eb79754f None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:50.770546 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc6d8a-752d-456b-9c95-a5c8eb79754f None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:50.770819 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc6d8a-752d-456b-9c95-a5c8eb79754f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:50.778844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc485a06-f190-4b99-8ef5-6e8aec371bf5 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:50.795247 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-4bbc6d8a-752d-456b-9c95-a5c8eb79754f None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:50.796484 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-4bbc6d8a-752d-456b-9c95-a5c8eb79754f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:50.821555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807f13d-6d5d-4d62-8b4f-8f92877b9963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:50.822847 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807f13d-6d5d-4d62-8b4f-8f92877b9963 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:50.823799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807f13d-6d5d-4d62-8b4f-8f92877b9963 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:50.824539 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807f13d-6d5d-4d62-8b4f-8f92877b9963 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:50.837994 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807f13d-6d5d-4d62-8b4f-8f92877b9963 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:50.839474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807f13d-6d5d-4d62-8b4f-8f92877b9963 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:50.855748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-9807f13d-6d5d-4d62-8b4f-8f92877b9963 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:50.855748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:50.855748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:50.855748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:50.855748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:50.855748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:50.855748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:50.855748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:50.855748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:50.855748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:50.855748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:50.855748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:50.855748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:50.855748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:50.855748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:50.855748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:50.856767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:50.856767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:50.856767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:50.856767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:50.856767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:50.856767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:50.856767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:50.856767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:50.856767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:50.856767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:50.856767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:50.856767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:50.856767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:50.856767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:50.856767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:50.856767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:50.856767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:50.857708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:50.857708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:50.857708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:50.857708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 320/636] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:50.857948 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807f13d-6d5d-4d62-8b4f-8f92877b9963 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:50.869768 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:50.880422 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 317/637] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:50.881730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494aa88-39f1-4cdc-9d7c-896d8dbbc69e None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:50.882023 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494aa88-39f1-4cdc-9d7c-896d8dbbc69e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:50.882298 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494aa88-39f1-4cdc-9d7c-896d8dbbc69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:50.906634 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e494aa88-39f1-4cdc-9d7c-896d8dbbc69e None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:50.907748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e494aa88-39f1-4cdc-9d7c-896d8dbbc69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:50.942777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:50.993809 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 318/638] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:50.995297 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494aa88-39f1-4cdc-9d7c-896d8dbbc69e None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:51.004257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807f13d-6d5d-4d62-8b4f-8f92877b9963 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:51.040426 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b466fb-6066-41ca-9afe-12b87293a0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:51.041050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b466fb-6066-41ca-9afe-12b87293a0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:51.041352 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b466fb-6066-41ca-9afe-12b87293a0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:51.041618 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b466fb-6066-41ca-9afe-12b87293a0b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:51.055178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b466fb-6066-41ca-9afe-12b87293a0b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:51.056607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b466fb-6066-41ca-9afe-12b87293a0b2 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:51.074162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-d5b466fb-6066-41ca-9afe-12b87293a0b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:51.074162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:51.074162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:51.074162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:51.074162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:51.074162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:51.074162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:51.074162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:51.074162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:51.074162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:51.074162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:51.074162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:51.074162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:51.074162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:51.074162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:51.074162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:51.075183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:51.075183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:51.075183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:51.075183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:51.075183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:51.075183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:51.075183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:51.075183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:51.075183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:51.075183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:51.075183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:51.075183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:51.075183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:51.075183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:51.075183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:51.075183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:51.075183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:51.076672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:51.076672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:51.076672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:51.076672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 321/639] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:51.081567 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b466fb-6066-41ca-9afe-12b87293a0b2 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:51.136816 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494aa88-39f1-4cdc-9d7c-896d8dbbc69e None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:51.145098 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:51.194572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9239b7b-fb30-486f-bd5d-bc76a77b93f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:51.195195 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9239b7b-fb30-486f-bd5d-bc76a77b93f4 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:51.195471 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9239b7b-fb30-486f-bd5d-bc76a77b93f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:51.195742 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9239b7b-fb30-486f-bd5d-bc76a77b93f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:51.196650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9239b7b-fb30-486f-bd5d-bc76a77b93f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:51.196956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9239b7b-fb30-486f-bd5d-bc76a77b93f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:51.199061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9239b7b-fb30-486f-bd5d-bc76a77b93f4 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:51.207388 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b466fb-6066-41ca-9afe-12b87293a0b2 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:51.214771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 319/640] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:51.216078 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9239b7b-fb30-486f-bd5d-bc76a77b93f4 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:51.244989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfb907-539c-464c-8346-ba84cc37395b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:51.245966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfb907-539c-464c-8346-ba84cc37395b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:51.246507 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfb907-539c-464c-8346-ba84cc37395b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:51.247463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cfb907-539c-464c-8346-ba84cc37395b None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:51.248613 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cfb907-539c-464c-8346-ba84cc37395b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:51.249122 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cfb907-539c-464c-8346-ba84cc37395b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:51.251396 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cfb907-539c-464c-8346-ba84cc37395b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:51.253799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:51.267076 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 322/641] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:51.272997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfb907-539c-464c-8346-ba84cc37395b None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:51.342838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9239b7b-fb30-486f-bd5d-bc76a77b93f4 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:51.383428 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1123a-b0a4-46c3-8c4b-d990427baf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:51.384077 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1123a-b0a4-46c3-8c4b-d990427baf69 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:51.384362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1123a-b0a4-46c3-8c4b-d990427baf69 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:51.384640 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1123a-b0a4-46c3-8c4b-d990427baf69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:51.391995 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cfb907-539c-464c-8346-ba84cc37395b None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:51.395276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1123a-b0a4-46c3-8c4b-d990427baf69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:51.401248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-9da1123a-b0a4-46c3-8c4b-d990427baf69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:51.401248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:51.401248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:51.401248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:51.401248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:51.401248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:51.401248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:51.401248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:51.401248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:51.401248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:51.401248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:51.401248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:51.401248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:51.401248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:51.401248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:51.401248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:51.402381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:51.402381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:51.402381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:51.402381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:51.402381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:51.402381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:51.402381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:51.402381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:51.402381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:51.402381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:51.402381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:51.402381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:51.402381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:51.402381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:51.402381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:51.402381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:51.403493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:51.403493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:51.403493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:51.403493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:51.403493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:51.403493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:51.403493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:51.403493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:51.403493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:51.403493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:51.403493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:51.403493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:51.407241 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 320/642] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:51.408075 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1123a-b0a4-46c3-8c4b-d990427baf69 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:51.414002 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:51.434772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b530e3c-bbde-41e5-a310-212c3582aa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:51.435408 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b530e3c-bbde-41e5-a310-212c3582aa8c None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:51.435684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b530e3c-bbde-41e5-a310-212c3582aa8c None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:51.435969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b530e3c-bbde-41e5-a310-212c3582aa8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:51.443220 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b530e3c-bbde-41e5-a310-212c3582aa8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:51.444648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b530e3c-bbde-41e5-a310-212c3582aa8c None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:51.457117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-6b530e3c-bbde-41e5-a310-212c3582aa8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:51.457117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:51.457117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:51.457117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:51.457117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:51.457117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:51.457117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:51.457117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:51.457117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:51.457117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:51.457117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:51.457117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:51.457117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:09:51.457117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:09:51.457117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:09:51.462825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:09:51.462825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:51.462825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:51.462825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:51.462825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:51.462825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:51.462825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:51.462825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:51.462825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:51.462825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:51.462825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:51.462825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:51.462825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:51.462825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:51.462825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:51.462825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:51.462825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:51.463981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:09:51.463981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:09:51.463981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:09:51.463981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:09:51.463981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:09:51.463981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:09:51.463981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:51.463981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:51.463981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 323/643] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:51.463981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b530e3c-bbde-41e5-a310-212c3582aa8c None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:51.508004 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:51.523219 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1123a-b0a4-46c3-8c4b-d990427baf69 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:51.557078 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9bf6b-f2aa-4674-bbb3-8f84e831ce8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:51.557725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9bf6b-f2aa-4674-bbb3-8f84e831ce8d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:51.558005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9bf6b-f2aa-4674-bbb3-8f84e831ce8d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:51.558270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9bf6b-f2aa-4674-bbb3-8f84e831ce8d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:51.566547 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa9bf6b-f2aa-4674-bbb3-8f84e831ce8d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:51.566681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b530e3c-bbde-41e5-a310-212c3582aa8c None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:51.567988 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa9bf6b-f2aa-4674-bbb3-8f84e831ce8d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:51.579292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-7aa9bf6b-f2aa-4674-bbb3-8f84e831ce8d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 12:09:51.579292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:51.579292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:51.579292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:51.579292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:51.579292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:51.579292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:51.579292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:51.579292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:51.579292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:51.579292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:51.579292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:51.579292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:09:51.579292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:09:51.579292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:09:51.579292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:51.580273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:51.581324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:51.581324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:51.581324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:09:51.581324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:09:51.581324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:09:51.581324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:09:51.581324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:09:51.581324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:09:51.581324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 12:09:51.581324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:51.581324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 321/644] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:51.581324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9bf6b-f2aa-4674-bbb3-8f84e831ce8d None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:51.607495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bcedc-540b-4a17-820e-b285bea2ce01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:51.608130 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bcedc-540b-4a17-820e-b285bea2ce01 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:51.608411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bcedc-540b-4a17-820e-b285bea2ce01 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:51.608684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bcedc-540b-4a17-820e-b285bea2ce01 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:51.620663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998bcedc-540b-4a17-820e-b285bea2ce01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:51.626865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-998bcedc-540b-4a17-820e-b285bea2ce01 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:51.626865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:51.626865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:51.626865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:51.626865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:51.626865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:51.626865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:51.626865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:51.626865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:51.626865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:51.626865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:51.626865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:51.626865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:51.626865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:51.626865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:51.626865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:51.627852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:51.627852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:51.627852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:51.627852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:51.627852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:51.627852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:51.627852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:51.627852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:51.627852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:51.627852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:51.627852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:51.627852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:51.627852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:51.627852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:51.627852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:51.627852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:51.628880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:51.628880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:51.628880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:51.628880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:51.628880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:51.628880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:51.628880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:51.628880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:51.628880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:51.628880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:51.628880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:51.628880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:51.628880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 324/645] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:51.628880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bcedc-540b-4a17-820e-b285bea2ce01 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:51.673681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:51.690596 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9bf6b-f2aa-4674-bbb3-8f84e831ce8d None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:51.735711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942c1c6-1803-4856-bec5-ff90f23f6e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:51.737089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942c1c6-1803-4856-bec5-ff90f23f6e3d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:51.737839 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942c1c6-1803-4856-bec5-ff90f23f6e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:51.738505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942c1c6-1803-4856-bec5-ff90f23f6e3d None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:51.739320 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942c1c6-1803-4856-bec5-ff90f23f6e3d None admin] RBAC: Adding query filter params (name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:51.739763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942c1c6-1803-4856-bec5-ff90f23f6e3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:51.740086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bcedc-540b-4a17-820e-b285bea2ce01 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:51.741619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942c1c6-1803-4856-bec5-ff90f23f6e3d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:51.752338 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 322/646] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:09:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:51.753580 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942c1c6-1803-4856-bec5-ff90f23f6e3d None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:51.769632 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:51.776090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd417b68-4547-48a9-ab22-c0c9d27a0a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:51.776940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd417b68-4547-48a9-ab22-c0c9d27a0a6a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:51.777325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd417b68-4547-48a9-ab22-c0c9d27a0a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:51.777673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd417b68-4547-48a9-ab22-c0c9d27a0a6a None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:51.778676 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd417b68-4547-48a9-ab22-c0c9d27a0a6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:51.779505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd417b68-4547-48a9-ab22-c0c9d27a0a6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:51.782148 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd417b68-4547-48a9-ab22-c0c9d27a0a6a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:51.801976 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 325/647] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:09:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:51.803062 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd417b68-4547-48a9-ab22-c0c9d27a0a6a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:51.871592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942c1c6-1803-4856-bec5-ff90f23f6e3d None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:51.909515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c37054-55dc-4246-86d5-d7211ef9b277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:51.910305 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c37054-55dc-4246-86d5-d7211ef9b277 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:51.910935 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:51.911642 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c37054-55dc-4246-86d5-d7211ef9b277 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:51.912337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c37054-55dc-4246-86d5-d7211ef9b277 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:51.913545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c37054-55dc-4246-86d5-d7211ef9b277 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:51.913920 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c37054-55dc-4246-86d5-d7211ef9b277 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:51.915000 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd417b68-4547-48a9-ab22-c0c9d27a0a6a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:51.915632 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c37054-55dc-4246-86d5-d7211ef9b277 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:51.944931 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 323/648] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Sep 28 12:09:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:51.944931 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c37054-55dc-4246-86d5-d7211ef9b277 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:51.950502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2ed37-06f5-4ef4-a229-e68c991e6c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:51.951107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2ed37-06f5-4ef4-a229-e68c991e6c4b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:51.951385 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2ed37-06f5-4ef4-a229-e68c991e6c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:51.951655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2ed37-06f5-4ef4-a229-e68c991e6c4b None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:51.952311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2ed37-06f5-4ef4-a229-e68c991e6c4b None admin] RBAC: Adding query filter params (name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:51.952611 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2ed37-06f5-4ef4-a229-e68c991e6c4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:51.954322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2ed37-06f5-4ef4-a229-e68c991e6c4b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:51.962496 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 326/649] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:09:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:51.967784 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2ed37-06f5-4ef4-a229-e68c991e6c4b None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:52.019964 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:52.058692 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c37054-55dc-4246-86d5-d7211ef9b277 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:52.071967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2ed37-06f5-4ef4-a229-e68c991e6c4b None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:52.098717 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eda1a7-528d-4e32-af7b-80864085a327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:52.099357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eda1a7-528d-4e32-af7b-80864085a327 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:52.099640 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eda1a7-528d-4e32-af7b-80864085a327 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:52.099950 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eda1a7-528d-4e32-af7b-80864085a327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:52.107177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12c90a-941a-454f-93c7-8bc0f2ea0d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:52.107811 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12c90a-941a-454f-93c7-8bc0f2ea0d30 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:52.108091 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12c90a-941a-454f-93c7-8bc0f2ea0d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:52.108750 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12c90a-941a-454f-93c7-8bc0f2ea0d30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:52.109505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12c90a-941a-454f-93c7-8bc0f2ea0d30 None admin] RBAC: Adding query filter params (role.id=60963384f3e54d9cbfbc8d12023a6cd9, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=982b82e8ac424318a934f9149fb91706) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:52.110314 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12c90a-941a-454f-93c7-8bc0f2ea0d30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:52.111426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eda1a7-528d-4e32-af7b-80864085a327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:52.111995 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12c90a-941a-454f-93c7-8bc0f2ea0d30 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:52.112856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eda1a7-528d-4e32-af7b-80864085a327 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:52.118195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 327/650] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:09:51 2022] GET /identity/v3/role_assignments?user.id=982b82e8ac424318a934f9149fb91706&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=60963384f3e54d9cbfbc8d12023a6cd9 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:52.118820 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12c90a-941a-454f-93c7-8bc0f2ea0d30 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:52.133463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-b1eda1a7-528d-4e32-af7b-80864085a327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:52.133463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:52.133463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:52.133463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:52.133463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:52.133463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:52.133463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:52.133463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:52.133463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:52.133463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:52.133463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:52.133463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:52.133463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:52.133463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:52.133463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:52.133463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:52.134505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:52.134505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:52.134505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:52.134505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:52.134505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:52.134505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:52.134505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:52.134505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:52.134505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:52.134505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:52.134505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:52.134505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:52.134505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:52.134505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:52.134505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:52.134505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:52.134505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:52.135420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:52.135420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:52.135420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:52.140167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 324/651] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:52.140948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eda1a7-528d-4e32-af7b-80864085a327 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:52.164390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:52.264321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eda1a7-528d-4e32-af7b-80864085a327 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:52.279431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12c90a-941a-454f-93c7-8bc0f2ea0d30 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:52.287026 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:52.305777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1eff17-595c-4136-965b-951b03339b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:52.306521 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1eff17-595c-4136-965b-951b03339b4c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:52.306912 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1eff17-595c-4136-965b-951b03339b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:52.307279 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1eff17-595c-4136-965b-951b03339b4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:52.319918 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1eff17-595c-4136-965b-951b03339b4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:52.321397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1eff17-595c-4136-965b-951b03339b4c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:52.325950 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3c5c2-4172-4aca-b10a-eb13bee57600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:52.326560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3c5c2-4172-4aca-b10a-eb13bee57600 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:52.326838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3c5c2-4172-4aca-b10a-eb13bee57600 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:52.327108 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3c5c2-4172-4aca-b10a-eb13bee57600 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:52.337638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-bd1eff17-595c-4136-965b-951b03339b4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:52.337638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:52.337638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:52.337638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:52.337638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:52.337638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:52.337638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:52.337638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:52.337638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:52.337638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:52.337638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:52.337638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:52.337638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:52.337638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:52.337638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:52.337638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:52.338612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:52.338612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:52.338612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:52.338612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:52.338612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:52.338612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:52.338612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:52.338612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:52.338612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:52.338612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:52.338612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:52.338612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:52.338612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:52.338612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:52.338612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:52.338612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:52.338612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:52.339501 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:52.339501 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:52.339501 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:52.343670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 325/652] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:52.344887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1eff17-595c-4136-965b-951b03339b4c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:52.412261 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3c5c2-4172-4aca-b10a-eb13bee57600 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:52.413946 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3c5c2-4172-4aca-b10a-eb13bee57600 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:52.443454 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:52.475758 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1eff17-595c-4136-965b-951b03339b4c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:52.522807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf3058-211e-429b-9c5d-3dc0e59bd756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:52.523435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf3058-211e-429b-9c5d-3dc0e59bd756 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:52.523753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf3058-211e-429b-9c5d-3dc0e59bd756 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:52.524029 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcf3058-211e-429b-9c5d-3dc0e59bd756 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:52.524676 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf3058-211e-429b-9c5d-3dc0e59bd756 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:52.524977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf3058-211e-429b-9c5d-3dc0e59bd756 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:52.526478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcf3058-211e-429b-9c5d-3dc0e59bd756 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:52.538008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 328/653] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:52.539229 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3c5c2-4172-4aca-b10a-eb13bee57600 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:52.544450 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 326/654] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:52.545842 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf3058-211e-429b-9c5d-3dc0e59bd756 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:52.579077 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:52.660346 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcf3058-211e-429b-9c5d-3dc0e59bd756 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:52.664355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3c5c2-4172-4aca-b10a-eb13bee57600 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:52.691176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:52.700480 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b630c0-2221-4ca8-a187-ab87fd1b6d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:52.701284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b630c0-2221-4ca8-a187-ab87fd1b6d21 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:52.701624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b630c0-2221-4ca8-a187-ab87fd1b6d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:52.701982 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b630c0-2221-4ca8-a187-ab87fd1b6d21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:52.704518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd030d3b-acad-4745-b79e-ccdc1410e47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:52.705356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd030d3b-acad-4745-b79e-ccdc1410e47a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:52.705823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd030d3b-acad-4745-b79e-ccdc1410e47a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:52.706276 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd030d3b-acad-4745-b79e-ccdc1410e47a None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:52.707114 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd030d3b-acad-4745-b79e-ccdc1410e47a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:52.707610 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd030d3b-acad-4745-b79e-ccdc1410e47a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:52.709208 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd030d3b-acad-4745-b79e-ccdc1410e47a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:52.714415 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b630c0-2221-4ca8-a187-ab87fd1b6d21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:52.715935 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b630c0-2221-4ca8-a187-ab87fd1b6d21 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:52.728273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 329/655] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:52.728273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd030d3b-acad-4745-b79e-ccdc1410e47a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:52.739008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-d6b630c0-2221-4ca8-a187-ab87fd1b6d21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:52.739008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:52.739008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:52.739008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:52.739008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:52.739008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:52.739008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:52.739008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:52.739008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:52.739008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:52.739008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:52.739008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:52.739008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:52.739008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:52.739008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:52.739008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:52.739980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:52.739980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:52.739980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:52.739980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:52.739980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:52.739980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:52.739980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:52.739980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:52.739980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:52.739980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:52.739980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:52.739980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:52.739980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:52.739980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:52.739980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:52.739980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:52.739980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:52.740851 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:52.740851 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:52.740851 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:52.740851 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 327/656] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:52.741545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b630c0-2221-4ca8-a187-ab87fd1b6d21 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:52.838150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:52.855936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd030d3b-acad-4745-b79e-ccdc1410e47a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:52.860108 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b630c0-2221-4ca8-a187-ab87fd1b6d21 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:52.896305 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53cc13-3ff1-4b70-b534-f72ff710cc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:52.897029 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53cc13-3ff1-4b70-b534-f72ff710cc6e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:52.897436 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53cc13-3ff1-4b70-b534-f72ff710cc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:52.897804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53cc13-3ff1-4b70-b534-f72ff710cc6e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:52.898600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599626f5-8ae6-4ae8-b395-64c940f1bd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:52.899435 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599626f5-8ae6-4ae8-b395-64c940f1bd93 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:52.900050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599626f5-8ae6-4ae8-b395-64c940f1bd93 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:52.900491 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599626f5-8ae6-4ae8-b395-64c940f1bd93 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:52.906031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53cc13-3ff1-4b70-b534-f72ff710cc6e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:52.908261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53cc13-3ff1-4b70-b534-f72ff710cc6e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:52.910301 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599626f5-8ae6-4ae8-b395-64c940f1bd93 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:52.912381 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599626f5-8ae6-4ae8-b395-64c940f1bd93 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:52.920304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-4a53cc13-3ff1-4b70-b534-f72ff710cc6e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 28 12:09:52.920304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:52.920304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:52.920304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:52.920304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:52.920304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:52.920304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:52.920304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:52.920304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:52.920304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:52.920304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:52.920304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:52.920304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:09:52.920304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:09:52.920304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:09:52.920304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:52.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:52.922568 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:52.922568 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:52.922568 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:09:52.922568 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:09:52.922568 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:09:52.922568 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:09:52.922568 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:09:52.922568 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:09:52.922568 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 28 12:09:52.922568 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:52.926412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 328/657] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:52.927337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53cc13-3ff1-4b70-b534-f72ff710cc6e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:52.927805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-599626f5-8ae6-4ae8-b395-64c940f1bd93 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 12:09:52.927805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:52.927805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:52.927805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:52.927805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:52.927805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:52.927805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:52.927805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:52.927805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:52.927805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:52.927805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:52.927805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:52.927805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:09:52.927805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:09:52.927805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:09:52.927805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:52.928685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:52.929689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:52.929689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:52.929689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:09:52.929689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:09:52.929689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:09:52.929689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:09:52.929689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:09:52.929689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:09:52.929689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 12:09:52.929689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:52.938063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 330/658] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:52.939653 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599626f5-8ae6-4ae8-b395-64c940f1bd93 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:53.000078 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:53.057200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53cc13-3ff1-4b70-b534-f72ff710cc6e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:53.065563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599626f5-8ae6-4ae8-b395-64c940f1bd93 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:53.098134 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf63bc3-e5a3-4771-a8b3-87930786cb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:53.098670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf63bc3-e5a3-4771-a8b3-87930786cb0a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:53.098948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf63bc3-e5a3-4771-a8b3-87930786cb0a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:53.099220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf63bc3-e5a3-4771-a8b3-87930786cb0a None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:53.099859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf63bc3-e5a3-4771-a8b3-87930786cb0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:53.100160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf63bc3-e5a3-4771-a8b3-87930786cb0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:53.101574 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf63bc3-e5a3-4771-a8b3-87930786cb0a None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:53.103338 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e811f-70fb-40a6-8ee1-a04fd7e703f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:53.104435 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e811f-70fb-40a6-8ee1-a04fd7e703f4 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:53.104877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e811f-70fb-40a6-8ee1-a04fd7e703f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:53.105329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e811f-70fb-40a6-8ee1-a04fd7e703f4 None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:53.106204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e811f-70fb-40a6-8ee1-a04fd7e703f4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:53.106662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e811f-70fb-40a6-8ee1-a04fd7e703f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:53.108236 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e811f-70fb-40a6-8ee1-a04fd7e703f4 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:53.114417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 329/659] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:53.115297 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf63bc3-e5a3-4771-a8b3-87930786cb0a None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:53.117923 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:53.133278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 331/660] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Sep 28 12:09:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:53.134842 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a57795-7db7-4bb3-889d-a1c567f686c8 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:53.135014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a57795-7db7-4bb3-889d-a1c567f686c8 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:53.135346 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a57795-7db7-4bb3-889d-a1c567f686c8 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:53.136698 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 332/661] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:53.137283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a57795-7db7-4bb3-889d-a1c567f686c8 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:53.229036 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf63bc3-e5a3-4771-a8b3-87930786cb0a None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:53.259064 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a57795-7db7-4bb3-889d-a1c567f686c8 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:53.260554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:53.263393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfda2f-6554-42fc-9640-ad0240c41db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:53.263996 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfda2f-6554-42fc-9640-ad0240c41db5 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:53.264276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfda2f-6554-42fc-9640-ad0240c41db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:53.264546 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dfda2f-6554-42fc-9640-ad0240c41db5 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:53.265290 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfda2f-6554-42fc-9640-ad0240c41db5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:53.265595 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfda2f-6554-42fc-9640-ad0240c41db5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:53.267259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dfda2f-6554-42fc-9640-ad0240c41db5 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:53.286410 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 330/662] 10.222.0.60 () {56 vars in 1139 bytes} [Wed Sep 28 12:09:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:53.287490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfda2f-6554-42fc-9640-ad0240c41db5 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:53.303511 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc777ff3-3b09-4d25-8fcf-6b2ca9004799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:53.304432 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc777ff3-3b09-4d25-8fcf-6b2ca9004799 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:53.304890 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc777ff3-3b09-4d25-8fcf-6b2ca9004799 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:53.305591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc777ff3-3b09-4d25-8fcf-6b2ca9004799 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:53.313418 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc777ff3-3b09-4d25-8fcf-6b2ca9004799 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:53.315275 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc777ff3-3b09-4d25-8fcf-6b2ca9004799 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:53.326942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-bc777ff3-3b09-4d25-8fcf-6b2ca9004799 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:53.326942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:53.326942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:53.326942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:53.326942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:53.326942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:53.326942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:53.326942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:53.326942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:53.326942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:53.326942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:53.326942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:53.326942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:09:53.326942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:09:53.326942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:09:53.327905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:09:53.327905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:53.327905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:53.327905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:53.327905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:53.327905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:53.327905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:53.327905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:53.327905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:53.327905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:53.327905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:53.327905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:53.327905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:53.327905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:53.327905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:53.327905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:53.327905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:53.329186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:09:53.329186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:09:53.329186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:09:53.329186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:09:53.329186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:09:53.329186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:09:53.329186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:53.329186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:53.331178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 333/663] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:53.332513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af247bc-65f3-4196-b22c-a337b43dd068 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:53.332790 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af247bc-65f3-4196-b22c-a337b43dd068 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:53.333059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af247bc-65f3-4196-b22c-a337b43dd068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:53.350830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:53.352116 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7af247bc-65f3-4196-b22c-a337b43dd068 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:53.353666 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7af247bc-65f3-4196-b22c-a337b43dd068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:53.395374 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dfda2f-6554-42fc-9640-ad0240c41db5 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:53.433446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 334/664] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:53.433446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af247bc-65f3-4196-b22c-a337b43dd068 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:53.440222 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed15f9-8582-44c5-b8b0-7f5aa578917b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:53.441039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed15f9-8582-44c5-b8b0-7f5aa578917b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:53.441496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed15f9-8582-44c5-b8b0-7f5aa578917b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:53.441928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed15f9-8582-44c5-b8b0-7f5aa578917b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:53.516274 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ed15f9-8582-44c5-b8b0-7f5aa578917b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:53.518408 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ed15f9-8582-44c5-b8b0-7f5aa578917b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:53.539716 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:53.550954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af247bc-65f3-4196-b22c-a337b43dd068 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:53.589298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747f918-f738-47d9-add8-1bf2d77e102a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:53.591129 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747f918-f738-47d9-add8-1bf2d77e102a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:53.591554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747f918-f738-47d9-add8-1bf2d77e102a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:53.592132 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747f918-f738-47d9-add8-1bf2d77e102a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:53.602234 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:53.665569 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 331/665] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:53.666732 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed15f9-8582-44c5-b8b0-7f5aa578917b None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:53.671693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747f918-f738-47d9-add8-1bf2d77e102a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:53.673245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747f918-f738-47d9-add8-1bf2d77e102a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:53.780002 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:53.783504 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed15f9-8582-44c5-b8b0-7f5aa578917b None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:53.794219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 335/666] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:53.795614 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747f918-f738-47d9-add8-1bf2d77e102a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:53.818861 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c82bd3-d77d-47b2-b822-061b194b72f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:53.819473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c82bd3-d77d-47b2-b822-061b194b72f9 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:53.819761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c82bd3-d77d-47b2-b822-061b194b72f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:53.820033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c82bd3-d77d-47b2-b822-061b194b72f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:53.820730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c82bd3-d77d-47b2-b822-061b194b72f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:53.821038 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c82bd3-d77d-47b2-b822-061b194b72f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:53.822520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c82bd3-d77d-47b2-b822-061b194b72f9 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:53.835539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 332/667] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:09:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:53.836600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c82bd3-d77d-47b2-b822-061b194b72f9 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:53.860231 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:53.911880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747f918-f738-47d9-add8-1bf2d77e102a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:53.945843 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c82bd3-d77d-47b2-b822-061b194b72f9 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:53.950874 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf98ab-1971-440d-93f9-794db6090542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:53.951502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf98ab-1971-440d-93f9-794db6090542 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:53.951776 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf98ab-1971-440d-93f9-794db6090542 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:53.952046 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecf98ab-1971-440d-93f9-794db6090542 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:53.963293 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf98ab-1971-440d-93f9-794db6090542 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:53.964723 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecf98ab-1971-440d-93f9-794db6090542 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:53.980455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-cecf98ab-1971-440d-93f9-794db6090542 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:53.980455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:53.980455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:53.980455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:53.980455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:53.980455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:53.980455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:53.980455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:53.980455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:53.980455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:53.980455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:53.980455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:53.980455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:53.980455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:53.980455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:53.980455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:53.981470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:53.981470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:53.981470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:53.981470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:53.981470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:53.981470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:53.981470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:53.981470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:53.981470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:53.981470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:53.981470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:53.981470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:53.981470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:53.981470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:53.981470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:53.981470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:53.981470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:53.982368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:53.982368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:53.982368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:53.982368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 336/668] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:53.982368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf98ab-1971-440d-93f9-794db6090542 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:53.986105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae3e516-fa35-448f-83a5-bd79968e9dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:53.986739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae3e516-fa35-448f-83a5-bd79968e9dab None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:53.987012 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae3e516-fa35-448f-83a5-bd79968e9dab None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:53.987278 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae3e516-fa35-448f-83a5-bd79968e9dab None admin] PATH_INFO: `/v3/roles/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:53.998423 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae3e516-fa35-448f-83a5-bd79968e9dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:54.004280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-4ae3e516-fa35-448f-83a5-bd79968e9dab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:54.004280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:54.004280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:54.004280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:54.004280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:54.004280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:54.004280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:54.004280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:54.004280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:54.004280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:54.004280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:54.004280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:54.004280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:54.004280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:54.004280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:54.004280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:54.005597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:54.005597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:54.005597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:54.005597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:54.005597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:54.005597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:54.005597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:54.005597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:54.005597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:54.005597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:54.005597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:54.005597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:54.005597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:54.005597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:54.005597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:54.005597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:54.006465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:54.006465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:54.006465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:54.006465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:54.006465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:54.006465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:54.006465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:54.006465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:54.006465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:54.006465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:54.006465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:09:54.006465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:54.006465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 333/669] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:09:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:54.007157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae3e516-fa35-448f-83a5-bd79968e9dab None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:54.040404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:54.094267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:54.099468 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecf98ab-1971-440d-93f9-794db6090542 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:54.114749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae3e516-fa35-448f-83a5-bd79968e9dab None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:54.138223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2dfad-b02d-41df-beb3-164c721f5961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:54.138899 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2dfad-b02d-41df-beb3-164c721f5961 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:54.139145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2dfad-b02d-41df-beb3-164c721f5961 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:54.139417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2dfad-b02d-41df-beb3-164c721f5961 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:54.151096 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2dfad-b02d-41df-beb3-164c721f5961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:54.152287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a24c6-af3c-460e-b87f-f180ca6b6c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:54.152543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2dfad-b02d-41df-beb3-164c721f5961 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:54.153309 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a24c6-af3c-460e-b87f-f180ca6b6c19 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:54.153772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a24c6-af3c-460e-b87f-f180ca6b6c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:54.154351 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a24c6-af3c-460e-b87f-f180ca6b6c19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:54.155372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a24c6-af3c-460e-b87f-f180ca6b6c19 None admin] RBAC: Adding query filter params (role.id=60963384f3e54d9cbfbc8d12023a6cd9, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=2354a4fd72cc42c58d6830eecb1e6253) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:54.155878 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a24c6-af3c-460e-b87f-f180ca6b6c19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:54.157742 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a24c6-af3c-460e-b87f-f180ca6b6c19 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:54.164616 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 334/670] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:09:54 2022] GET /identity/v3/role_assignments?user.id=2354a4fd72cc42c58d6830eecb1e6253&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=60963384f3e54d9cbfbc8d12023a6cd9 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:54.165710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a24c6-af3c-460e-b87f-f180ca6b6c19 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:54.167411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-b9a2dfad-b02d-41df-beb3-164c721f5961 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:54.167411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:54.167411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:54.167411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:54.167411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:54.167411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:54.167411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:54.167411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:54.167411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:54.167411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:54.167411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:54.167411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:54.167411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:54.167411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:54.167411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:54.167411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:54.168339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:54.168339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:54.168339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:54.168339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:54.168339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:54.168339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:54.168339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:54.168339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:54.168339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:54.168339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:54.168339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:54.168339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:54.168339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:54.168339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:54.168339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:54.168339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:54.168339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:54.169275 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:54.169275 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:54.169275 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:54.173632 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 337/671] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:54.174668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2dfad-b02d-41df-beb3-164c721f5961 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:54.322677 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:54.323241 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a24c6-af3c-460e-b87f-f180ca6b6c19 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:54.331241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2dfad-b02d-41df-beb3-164c721f5961 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:54.374470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158c144-f21d-4de6-a8f1-26ae828e4e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:54.375024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158c144-f21d-4de6-a8f1-26ae828e4e77 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:54.375883 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158c144-f21d-4de6-a8f1-26ae828e4e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:54.376163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158c144-f21d-4de6-a8f1-26ae828e4e77 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:54.376825 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158c144-f21d-4de6-a8f1-26ae828e4e77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:54.377154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158c144-f21d-4de6-a8f1-26ae828e4e77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:54.378553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1158c144-f21d-4de6-a8f1-26ae828e4e77 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:54.381495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42b791-4688-470b-8462-eeebfae90804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:54.382125 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42b791-4688-470b-8462-eeebfae90804 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:54.382405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42b791-4688-470b-8462-eeebfae90804 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:54.382675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42b791-4688-470b-8462-eeebfae90804 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:54.383366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42b791-4688-470b-8462-eeebfae90804 None admin] RBAC: Adding query filter params (name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:54.383663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42b791-4688-470b-8462-eeebfae90804 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:54.385082 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42b791-4688-470b-8462-eeebfae90804 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:54.396966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 338/672] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:09:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:54.398335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42b791-4688-470b-8462-eeebfae90804 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:54.405906 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 335/673] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:54.407007 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158c144-f21d-4de6-a8f1-26ae828e4e77 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:54.411185 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:54.519622 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42b791-4688-470b-8462-eeebfae90804 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:54.542375 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158c144-f21d-4de6-a8f1-26ae828e4e77 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:54.556642 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e9011-f781-43a4-8abe-37b231e1309c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:54.557293 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e9011-f781-43a4-8abe-37b231e1309c None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:54.557574 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e9011-f781-43a4-8abe-37b231e1309c None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:54.557884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e9011-f781-43a4-8abe-37b231e1309c None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:54.558476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e9011-f781-43a4-8abe-37b231e1309c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:54.559033 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e9011-f781-43a4-8abe-37b231e1309c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:54.560141 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e9011-f781-43a4-8abe-37b231e1309c None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:54.573965 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 339/674] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:54.578045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e9011-f781-43a4-8abe-37b231e1309c None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:54.585969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7fe3e-28f2-4cac-836e-da4be3f63351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:54.586928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7fe3e-28f2-4cac-836e-da4be3f63351 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:54.587393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7fe3e-28f2-4cac-836e-da4be3f63351 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:54.588383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7fe3e-28f2-4cac-836e-da4be3f63351 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:54.598414 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:54.602359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7fe3e-28f2-4cac-836e-da4be3f63351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:54.604075 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7fe3e-28f2-4cac-836e-da4be3f63351 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:54.624709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-8be7fe3e-28f2-4cac-836e-da4be3f63351 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:54.624709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:54.624709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:54.624709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:54.624709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:54.624709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:54.624709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:54.624709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:54.624709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:54.624709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:54.624709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:54.624709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:54.624709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:54.624709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:54.624709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:54.624709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:54.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:54.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:54.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:54.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:54.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:54.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:54.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:54.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:54.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:54.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:54.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:54.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:54.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:54.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:54.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:54.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:54.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:54.627623 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:54.627623 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:54.627623 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:54.629288 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 336/675] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:54.630526 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf204415-315c-47f1-80ae-41fd97df0560 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:54.630807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf204415-315c-47f1-80ae-41fd97df0560 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:54.631075 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf204415-315c-47f1-80ae-41fd97df0560 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:54.632314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 337/676] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:54.633000 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf204415-315c-47f1-80ae-41fd97df0560 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:54.689046 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e9011-f781-43a4-8abe-37b231e1309c None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:54.697283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:54.723033 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e989e2-f17f-4467-a3ba-dde2f0126c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:54.723669 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e989e2-f17f-4467-a3ba-dde2f0126c09 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:54.723946 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e989e2-f17f-4467-a3ba-dde2f0126c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:54.724217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e989e2-f17f-4467-a3ba-dde2f0126c09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:54.731060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e989e2-f17f-4467-a3ba-dde2f0126c09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:54.732475 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e989e2-f17f-4467-a3ba-dde2f0126c09 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:54.744630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-42e989e2-f17f-4467-a3ba-dde2f0126c09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:54.744630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:54.744630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:54.744630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:54.744630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:54.744630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:54.744630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:54.744630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:54.744630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:54.744630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:54.744630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:54.744630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:54.744630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:09:54.744630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:09:54.744630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:09:54.745584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:09:54.745584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:54.745584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:54.745584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:54.745584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:54.745584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:54.745584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:54.745584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:54.745584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:54.745584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:54.745584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:54.745584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:54.745584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:54.745584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:54.745584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:54.745584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:54.745584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:54.746473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:09:54.746473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:09:54.746473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:09:54.746473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:09:54.746473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:09:54.746473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:09:54.746473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:54.746473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:54.746473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 340/677] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:54.746473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e989e2-f17f-4467-a3ba-dde2f0126c09 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:54.751066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf204415-315c-47f1-80ae-41fd97df0560 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:54.786676 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d06cb6-0b0b-454b-a0ac-c28e0d80ddfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:54.787303 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d06cb6-0b0b-454b-a0ac-c28e0d80ddfa None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:54.787579 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d06cb6-0b0b-454b-a0ac-c28e0d80ddfa None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:54.787844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d06cb6-0b0b-454b-a0ac-c28e0d80ddfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:54.795665 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d06cb6-0b0b-454b-a0ac-c28e0d80ddfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:54.797681 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d06cb6-0b0b-454b-a0ac-c28e0d80ddfa None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:54.809670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-f7d06cb6-0b0b-454b-a0ac-c28e0d80ddfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:54.809670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:54.809670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:54.809670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:54.809670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:54.809670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:54.809670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:54.809670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:54.809670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:54.809670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:54.809670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:54.809670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:54.809670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:09:54.809670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:09:54.809670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:09:54.810577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:09:54.810577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:54.810577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:54.810577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:54.810577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:54.810577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:54.810577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:54.810577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:54.810577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:54.810577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:54.810577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:54.810577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:54.810577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:54.810577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:54.810577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:54.810577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:54.810577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:54.811475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:09:54.811475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:09:54.811475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:09:54.811475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:09:54.811475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:09:54.811475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:09:54.811475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:54.811475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:54.811475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 338/678] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:54.812489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f2670-01de-479e-9d6a-773d60bcd9c8 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:54.812489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f2670-01de-479e-9d6a-773d60bcd9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:54.812489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f2670-01de-479e-9d6a-773d60bcd9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:54.831288 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-395f2670-01de-479e-9d6a-773d60bcd9c8 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:54.832569 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-395f2670-01de-479e-9d6a-773d60bcd9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:54.835597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:54.862157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e989e2-f17f-4467-a3ba-dde2f0126c09 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:54.898797 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7b9b5-c423-46d3-b6a9-0b720bcfbeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:54.899640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7b9b5-c423-46d3-b6a9-0b720bcfbeea None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:54.900066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7b9b5-c423-46d3-b6a9-0b720bcfbeea None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:54.900501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7b9b5-c423-46d3-b6a9-0b720bcfbeea None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:54.901317 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7b9b5-c423-46d3-b6a9-0b720bcfbeea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:54.901875 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7b9b5-c423-46d3-b6a9-0b720bcfbeea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:54.903456 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7b9b5-c423-46d3-b6a9-0b720bcfbeea None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:54.903784 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 339/679] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:54.904621 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f2670-01de-479e-9d6a-773d60bcd9c8 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:54.926522 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 341/680] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:54.927282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7b9b5-c423-46d3-b6a9-0b720bcfbeea None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:54.960137 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:55.032890 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f2670-01de-479e-9d6a-773d60bcd9c8 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:55.044292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7b9b5-c423-46d3-b6a9-0b720bcfbeea None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:55.081101 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3613b5-8799-4e89-8e9a-7e2a45a6cf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:55.082097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3613b5-8799-4e89-8e9a-7e2a45a6cf32 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:55.082586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3613b5-8799-4e89-8e9a-7e2a45a6cf32 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:55.083286 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3613b5-8799-4e89-8e9a-7e2a45a6cf32 None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:55.084751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3613b5-8799-4e89-8e9a-7e2a45a6cf32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:55.089263 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3613b5-8799-4e89-8e9a-7e2a45a6cf32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:55.090755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3613b5-8799-4e89-8e9a-7e2a45a6cf32 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:55.101813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9201dc-fe73-4e3b-a0ba-aa69128962f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:55.102423 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9201dc-fe73-4e3b-a0ba-aa69128962f4 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:55.102698 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9201dc-fe73-4e3b-a0ba-aa69128962f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:55.102965 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9201dc-fe73-4e3b-a0ba-aa69128962f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:55.103647 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9201dc-fe73-4e3b-a0ba-aa69128962f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:55.103946 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9201dc-fe73-4e3b-a0ba-aa69128962f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:55.105499 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9201dc-fe73-4e3b-a0ba-aa69128962f4 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:55.121167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 340/681] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:09:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:55.121167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3613b5-8799-4e89-8e9a-7e2a45a6cf32 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:55.122134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:55.135362 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 342/682] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:09:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:55.136061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9201dc-fe73-4e3b-a0ba-aa69128962f4 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:55.268611 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3613b5-8799-4e89-8e9a-7e2a45a6cf32 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:55.276126 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:55.304400 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d2d3c-6751-4f66-8e48-dba0bdc26bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:55.304705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9201dc-fe73-4e3b-a0ba-aa69128962f4 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:55.305897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d2d3c-6751-4f66-8e48-dba0bdc26bab None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:55.306381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d2d3c-6751-4f66-8e48-dba0bdc26bab None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:55.306955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d2d3c-6751-4f66-8e48-dba0bdc26bab None admin] PATH_INFO: `/v3/users/swift` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:55.314966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d2d3c-6751-4f66-8e48-dba0bdc26bab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:55.316684 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d2d3c-6751-4f66-8e48-dba0bdc26bab None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:55.329047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-0c0d2d3c-6751-4f66-8e48-dba0bdc26bab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 12:09:55.329047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:55.329047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:55.329047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:55.329047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:55.329047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:55.329047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:55.329047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:55.329047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:55.329047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:55.329047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:55.329047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:55.329047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:09:55.329047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:09:55.329047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:09:55.329047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:55.330271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:55.331211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:55.331211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:55.331211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:09:55.331211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:09:55.331211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:09:55.331211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:09:55.331211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:09:55.331211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:09:55.331211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 12:09:55.331211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:55.340398 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 341/683] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:09:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:55.341519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136f3c7-ce8e-426e-bf3a-24f980896718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:55.341685 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d2d3c-6751-4f66-8e48-dba0bdc26bab None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:55.342125 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136f3c7-ce8e-426e-bf3a-24f980896718 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:55.342400 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136f3c7-ce8e-426e-bf3a-24f980896718 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:55.342667 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4136f3c7-ce8e-426e-bf3a-24f980896718 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:55.343385 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4136f3c7-ce8e-426e-bf3a-24f980896718 None admin] RBAC: Adding query filter params (role.id=60963384f3e54d9cbfbc8d12023a6cd9, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=8e08e00bcb5f47ffab58974ba376938a) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:55.343688 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4136f3c7-ce8e-426e-bf3a-24f980896718 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:55.346141 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4136f3c7-ce8e-426e-bf3a-24f980896718 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:55.353801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 343/684] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:09:55 2022] GET /identity/v3/role_assignments?user.id=8e08e00bcb5f47ffab58974ba376938a&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=60963384f3e54d9cbfbc8d12023a6cd9 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:55.354598 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136f3c7-ce8e-426e-bf3a-24f980896718 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:55.432684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:55.485344 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4136f3c7-ce8e-426e-bf3a-24f980896718 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:55.536325 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d2d3c-6751-4f66-8e48-dba0bdc26bab None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:55.574415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ded7f2-b22a-457c-b4e2-edee0c5b093f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:55.575365 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ded7f2-b22a-457c-b4e2-edee0c5b093f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:55.575365 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ded7f2-b22a-457c-b4e2-edee0c5b093f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:55.575658 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ded7f2-b22a-457c-b4e2-edee0c5b093f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:55.593955 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ded7f2-b22a-457c-b4e2-edee0c5b093f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:09:55.599696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-62ded7f2-b22a-457c-b4e2-edee0c5b093f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:09:55.599696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:55.599696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:55.599696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:55.599696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:55.599696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:55.599696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:55.599696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:55.599696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:55.599696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:55.599696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:55.599696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:55.599696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:09:55.599696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:09:55.599696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:09:55.599696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:09:55.600633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:09:55.600633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:09:55.600633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:09:55.600633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:09:55.600633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:55.600633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:55.600633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:55.600633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:55.600633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:55.600633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:55.600633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:55.600633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:55.600633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:55.600633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:55.600633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:55.600633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:55.601654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:55.601654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:55.601654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:55.601654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:55.601654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:09:55.601654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:09:55.601654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:09:55.601654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:09:55.601654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:09:55.601654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:09:55.601654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:09:55.601654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:55.601654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 344/685] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:09:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:55.602349 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ded7f2-b22a-457c-b4e2-edee0c5b093f None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:55.602418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26da15-de62-4ea9-84f3-e50a62422802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:55.603019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26da15-de62-4ea9-84f3-e50a62422802 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:55.603019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26da15-de62-4ea9-84f3-e50a62422802 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:55.604419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26da15-de62-4ea9-84f3-e50a62422802 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:55.604419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26da15-de62-4ea9-84f3-e50a62422802 None admin] RBAC: Adding query filter params (role.id=60963384f3e54d9cbfbc8d12023a6cd9, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=ba75586d3a7d4278ad6c0c31a1bed3df) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:55.604419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26da15-de62-4ea9-84f3-e50a62422802 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:55.605894 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26da15-de62-4ea9-84f3-e50a62422802 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:55.613189 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 342/686] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:09:55 2022] GET /identity/v3/role_assignments?user.id=ba75586d3a7d4278ad6c0c31a1bed3df&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=60963384f3e54d9cbfbc8d12023a6cd9 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:55.613938 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26da15-de62-4ea9-84f3-e50a62422802 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:55.660440 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:55.725182 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ded7f2-b22a-457c-b4e2-edee0c5b093f None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:55.730568 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26da15-de62-4ea9-84f3-e50a62422802 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:55.754500 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:55.770593 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87600150-a95e-423e-8931-d1a0f454c393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:55.771223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87600150-a95e-423e-8931-d1a0f454c393 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:55.771504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87600150-a95e-423e-8931-d1a0f454c393 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:55.771777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87600150-a95e-423e-8931-d1a0f454c393 None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:55.772515 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87600150-a95e-423e-8931-d1a0f454c393 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:55.772822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87600150-a95e-423e-8931-d1a0f454c393 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:55.774032 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8c3ba-3ccd-46e3-a199-dc6cd4338280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:55.774658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8c3ba-3ccd-46e3-a199-dc6cd4338280 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:55.774947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8c3ba-3ccd-46e3-a199-dc6cd4338280 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:55.775226 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8c3ba-3ccd-46e3-a199-dc6cd4338280 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:55.775503 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87600150-a95e-423e-8931-d1a0f454c393 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:55.775906 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8c3ba-3ccd-46e3-a199-dc6cd4338280 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:55.776216 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8c3ba-3ccd-46e3-a199-dc6cd4338280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:55.777835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8c3ba-3ccd-46e3-a199-dc6cd4338280 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:55.792420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 343/687] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:09:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:55.798464 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8c3ba-3ccd-46e3-a199-dc6cd4338280 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:55.799183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 345/688] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Sep 28 12:09:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:55.808451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87600150-a95e-423e-8931-d1a0f454c393 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:55.938352 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87600150-a95e-423e-8931-d1a0f454c393 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:55.940488 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:55.947062 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8c3ba-3ccd-46e3-a199-dc6cd4338280 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:55.987306 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d146936-8946-4c3e-a8b9-64dbe9758ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:55.987839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d146936-8946-4c3e-a8b9-64dbe9758ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:55.988119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d146936-8946-4c3e-a8b9-64dbe9758ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:55.988389 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d146936-8946-4c3e-a8b9-64dbe9758ac3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:55.990651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c06bc-c5f9-4f76-ad1a-5b942d1093a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:55.991968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c06bc-c5f9-4f76-ad1a-5b942d1093a7 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:55.992453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c06bc-c5f9-4f76-ad1a-5b942d1093a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:55.992935 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c06bc-c5f9-4f76-ad1a-5b942d1093a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:56.004821 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c06bc-c5f9-4f76-ad1a-5b942d1093a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:56.006759 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c06bc-c5f9-4f76-ad1a-5b942d1093a7 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:56.024090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-c41c06bc-c5f9-4f76-ad1a-5b942d1093a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:56.024090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:56.024090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:56.024090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:56.024090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:56.024090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:56.024090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:56.024090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:56.024090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:56.024090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:56.024090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:56.024090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:56.024090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:56.024090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:56.024090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:56.024090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:56.025069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:56.025069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:56.025069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:56.025069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:56.025069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:56.025069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:56.025069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:56.025069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:56.025069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:56.025069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:56.025069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:56.025069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:56.025069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:56.025069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:56.025069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:56.025069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:56.025069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:56.026281 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:56.026281 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:56.026281 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:56.028250 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 344/689] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:56.033489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c06bc-c5f9-4f76-ad1a-5b942d1093a7 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:56.062323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:56.106112 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d146936-8946-4c3e-a8b9-64dbe9758ac3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:56.107972 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d146936-8946-4c3e-a8b9-64dbe9758ac3 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:56.196309 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c06bc-c5f9-4f76-ad1a-5b942d1093a7 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:56.242284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc8bd6-1397-48b9-ae79-07a20acd22b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:56.243307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc8bd6-1397-48b9-ae79-07a20acd22b1 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:56.243877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc8bd6-1397-48b9-ae79-07a20acd22b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:56.244366 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc8bd6-1397-48b9-ae79-07a20acd22b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:56.245276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc8bd6-1397-48b9-ae79-07a20acd22b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:56.246012 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc8bd6-1397-48b9-ae79-07a20acd22b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:56.248366 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc8bd6-1397-48b9-ae79-07a20acd22b1 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:56.263306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 345/690] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:56.268826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc8bd6-1397-48b9-ae79-07a20acd22b1 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:56.274234 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:56.340668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 346/691] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:56.341959 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d146936-8946-4c3e-a8b9-64dbe9758ac3 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:56.456274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:56.481419 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d146936-8946-4c3e-a8b9-64dbe9758ac3 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:56.495578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc8bd6-1397-48b9-ae79-07a20acd22b1 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:56.527780 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16feb4-f954-4972-a44d-9e2997c0e60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:56.528429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16feb4-f954-4972-a44d-9e2997c0e60c None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:56.528709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16feb4-f954-4972-a44d-9e2997c0e60c None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:56.528982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16feb4-f954-4972-a44d-9e2997c0e60c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:56.549582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16feb4-f954-4972-a44d-9e2997c0e60c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:56.551397 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16feb4-f954-4972-a44d-9e2997c0e60c None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:56.568646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-5c16feb4-f954-4972-a44d-9e2997c0e60c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:56.568646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:56.568646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:56.568646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:56.568646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:56.568646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:56.568646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:56.568646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:56.568646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:56.568646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:56.568646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:56.568646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:56.568646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:56.568646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:56.568646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:56.568646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:56.569764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:56.569764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:56.569764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:56.569764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:56.569764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:56.569764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:56.569764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:56.569764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:56.569764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:56.569764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:56.569764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:56.569764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:56.569764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:56.569764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:56.569764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:56.569764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:56.569764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:56.570662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:56.570662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:56.570662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:56.572728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 347/692] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:56.573843 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bc395-7bec-4551-8ee9-30314a19f4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:56.574500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bc395-7bec-4551-8ee9-30314a19f4ea None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:56.574782 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bc395-7bec-4551-8ee9-30314a19f4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:56.575057 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bc395-7bec-4551-8ee9-30314a19f4ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:56.577943 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16feb4-f954-4972-a44d-9e2997c0e60c None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:56.584333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bc395-7bec-4551-8ee9-30314a19f4ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:56.585855 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80bc395-7bec-4551-8ee9-30314a19f4ea None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:56.614131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-d80bc395-7bec-4551-8ee9-30314a19f4ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:09:56.614131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:56.614131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:56.614131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:56.614131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:56.614131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:56.614131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:56.614131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:56.614131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:56.614131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:56.614131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:56.614131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:56.614131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:09:56.614131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:09:56.614131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:09:56.614131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:56.615148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:56.616084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:56.616084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:56.616084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:09:56.616084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:09:56.616084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:09:56.616084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:09:56.616084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:09:56.616084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:09:56.616084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:09:56.616084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:56.616084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 346/693] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 12:09:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:56.617232 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25c025-c9b1-4a13-90d6-8631b211eda2 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:56.617232 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25c025-c9b1-4a13-90d6-8631b211eda2 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:56.617410 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b25c025-c9b1-4a13-90d6-8631b211eda2 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:56.618831 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 347/694] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:56.626261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25c025-c9b1-4a13-90d6-8631b211eda2 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:56.694600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16feb4-f954-4972-a44d-9e2997c0e60c None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:56.701182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:56.736683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:56.745021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3ab93-a20f-48b2-9f8e-12c43cdd9065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:56.746076 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3ab93-a20f-48b2-9f8e-12c43cdd9065 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:56.746755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3ab93-a20f-48b2-9f8e-12c43cdd9065 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:56.747639 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3ab93-a20f-48b2-9f8e-12c43cdd9065 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:56.748754 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3ab93-a20f-48b2-9f8e-12c43cdd9065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:56.749293 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3ab93-a20f-48b2-9f8e-12c43cdd9065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:56.750976 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3ab93-a20f-48b2-9f8e-12c43cdd9065 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:56.751089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b25c025-c9b1-4a13-90d6-8631b211eda2 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:56.765202 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 348/695] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:56.768201 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876822f2-4b76-41b4-81bd-06028d26e1b9 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:56.768692 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876822f2-4b76-41b4-81bd-06028d26e1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:56.769244 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876822f2-4b76-41b4-81bd-06028d26e1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:56.789953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49414c08-a8c4-42f4-9060-0f152fad3006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:56.790681 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49414c08-a8c4-42f4-9060-0f152fad3006 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:56.791008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49414c08-a8c4-42f4-9060-0f152fad3006 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:56.791420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49414c08-a8c4-42f4-9060-0f152fad3006 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:56.792249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49414c08-a8c4-42f4-9060-0f152fad3006 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:56.792558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49414c08-a8c4-42f4-9060-0f152fad3006 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:56.793356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-876822f2-4b76-41b4-81bd-06028d26e1b9 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:56.794109 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49414c08-a8c4-42f4-9060-0f152fad3006 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:56.795000 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-876822f2-4b76-41b4-81bd-06028d26e1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:56.817564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 348/696] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 12:09:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:56.817564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49414c08-a8c4-42f4-9060-0f152fad3006 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:56.885157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 349/697] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:56.885157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876822f2-4b76-41b4-81bd-06028d26e1b9 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:56.940170 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49414c08-a8c4-42f4-9060-0f152fad3006 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:56.963770 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:56.980650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b5dec-edcc-4fff-8bbf-707519db9623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:56.981313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b5dec-edcc-4fff-8bbf-707519db9623 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:56.981595 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b5dec-edcc-4fff-8bbf-707519db9623 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:56.981874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b5dec-edcc-4fff-8bbf-707519db9623 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:56.998109 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b5dec-edcc-4fff-8bbf-707519db9623 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:57.000565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b5dec-edcc-4fff-8bbf-707519db9623 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:57.022225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-408b5dec-edcc-4fff-8bbf-707519db9623 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:57.022225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:57.022225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:57.022225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:57.022225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:57.022225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:57.022225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:57.022225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:57.022225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:57.022225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:57.022225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:57.022225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:57.022225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:09:57.022225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:09:57.022225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:09:57.023136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:09:57.023136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:57.023136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:57.023136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:57.023136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:57.023136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:57.023136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:57.023136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:57.023136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:57.023136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:57.023136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:57.023136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:57.023136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:57.023136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:57.023136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:57.023136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:57.023136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:57.024016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:09:57.024016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:09:57.024016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:09:57.024016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:09:57.024016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:09:57.024016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:09:57.024016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:57.024016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:57.024872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 349/698] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:57.030275 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b5dec-edcc-4fff-8bbf-707519db9623 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:57.064905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876822f2-4b76-41b4-81bd-06028d26e1b9 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:57.090751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:57.112453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce6c5c-509e-42e4-af7b-896697b41b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:57.113507 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce6c5c-509e-42e4-af7b-896697b41b45 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:57.114062 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce6c5c-509e-42e4-af7b-896697b41b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:57.114556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce6c5c-509e-42e4-af7b-896697b41b45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:57.208351 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b5dec-edcc-4fff-8bbf-707519db9623 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:57.219141 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce6c5c-509e-42e4-af7b-896697b41b45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:57.220884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce6c5c-509e-42e4-af7b-896697b41b45 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:57.245777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d3083-d9ae-43e0-913e-cd5115c17568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:57.246498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d3083-d9ae-43e0-913e-cd5115c17568 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:57.246811 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d3083-d9ae-43e0-913e-cd5115c17568 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:57.247154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d3083-d9ae-43e0-913e-cd5115c17568 None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:57.247858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d3083-d9ae-43e0-913e-cd5115c17568 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:57.248209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d3083-d9ae-43e0-913e-cd5115c17568 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:57.250052 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d3083-d9ae-43e0-913e-cd5115c17568 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:57.263276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 350/699] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:09:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:57.269726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d3083-d9ae-43e0-913e-cd5115c17568 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:57.325837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:57.338700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 350/700] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:09:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 12:09:57.340432 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce6c5c-509e-42e4-af7b-896697b41b45 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:57.370805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:57.399910 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d3083-d9ae-43e0-913e-cd5115c17568 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:57.446631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1671171-6888-4685-b4c7-ba154d15a266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:57.447764 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1671171-6888-4685-b4c7-ba154d15a266 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:57.448552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1671171-6888-4685-b4c7-ba154d15a266 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:57.449398 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1671171-6888-4685-b4c7-ba154d15a266 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:57.450166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1671171-6888-4685-b4c7-ba154d15a266 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:57.452503 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1671171-6888-4685-b4c7-ba154d15a266 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:57.458034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce6c5c-509e-42e4-af7b-896697b41b45 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:57.467034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-c1671171-6888-4685-b4c7-ba154d15a266 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 12:09:57.467034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:57.467034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:09:57.467034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:09:57.467034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:09:57.467034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:09:57.467034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:09:57.467034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:09:57.467034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 12:09:57.467034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:57.467034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:09:57.467034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:57.467034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:57.467034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:57.467034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:57.467034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:57.467034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:57.468041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:57.468041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:57.468041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:57.468041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:57.468041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:57.468041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:57.468041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:09:57.468041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:09:57.468041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:09:57.468041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:09:57.468041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:57.468041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:57.468041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:57.468041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:57.468041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:57.468041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:57.468946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:57.468946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:57.468946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:57.468946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:57.468946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:57.468946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:57.468946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:57.468946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:57.468946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:57.468946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:57.468946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:09:57.468946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:09:57.468946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 12:09:57.468946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:57.473324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 351/701] 10.222.0.60 () {56 vars in 1129 bytes} [Wed Sep 28 12:09:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:57.474361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1671171-6888-4685-b4c7-ba154d15a266 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:57.502914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a086f-396d-4c58-bedc-8bd696287a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:57.503783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a086f-396d-4c58-bedc-8bd696287a2d None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:57.503899 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a086f-396d-4c58-bedc-8bd696287a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:57.504134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a086f-396d-4c58-bedc-8bd696287a2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:57.504867 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a086f-396d-4c58-bedc-8bd696287a2d None admin] RBAC: Adding query filter params (role.id=60963384f3e54d9cbfbc8d12023a6cd9, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=975a8d7349994520bcbfb5f75e52aac7) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:57.505211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a086f-396d-4c58-bedc-8bd696287a2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:57.506737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a086f-396d-4c58-bedc-8bd696287a2d None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:57.518275 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 351/702] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:09:57 2022] GET /identity/v3/role_assignments?user.id=975a8d7349994520bcbfb5f75e52aac7&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=60963384f3e54d9cbfbc8d12023a6cd9 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:57.518275 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a086f-396d-4c58-bedc-8bd696287a2d None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:57.675456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1671171-6888-4685-b4c7-ba154d15a266 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:57.688465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:57.697303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:57.730061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a086f-396d-4c58-bedc-8bd696287a2d None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:57.751855 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a7c9c-3360-44fb-b6c0-76da5cab4fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:57.752512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a7c9c-3360-44fb-b6c0-76da5cab4fce None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:57.752795 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a7c9c-3360-44fb-b6c0-76da5cab4fce None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:57.753125 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a7c9c-3360-44fb-b6c0-76da5cab4fce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:57.777346 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a7c9c-3360-44fb-b6c0-76da5cab4fce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:57.782268 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a7c9c-3360-44fb-b6c0-76da5cab4fce None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:57.800070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479a06a-19ad-4817-a9e5-9be3223d0476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:57.800070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479a06a-19ad-4817-a9e5-9be3223d0476 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:57.800375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479a06a-19ad-4817-a9e5-9be3223d0476 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:57.800588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479a06a-19ad-4817-a9e5-9be3223d0476 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:57.801998 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479a06a-19ad-4817-a9e5-9be3223d0476 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:57.801998 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479a06a-19ad-4817-a9e5-9be3223d0476 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:57.805831 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479a06a-19ad-4817-a9e5-9be3223d0476 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:57.808478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-969a7c9c-3360-44fb-b6c0-76da5cab4fce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:57.808478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:57.808478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:57.808478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:57.808478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:57.808478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:57.808478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:57.808478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:57.808478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:57.808478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:57.808478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:57.808478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:57.808478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:57.808478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:57.808478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:57.808478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:57.809456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:57.809456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:57.809456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:57.809456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:57.809456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:57.809456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:57.809456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:57.809456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:57.809456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:57.809456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:57.809456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:57.809456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:57.809456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:57.809456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:57.809456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:57.809456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:57.809456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:57.810341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:57.810341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:57.810341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:57.810696 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 352/703] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:57.814195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 352/704] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Sep 28 12:09:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:57.818000 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a7c9c-3360-44fb-b6c0-76da5cab4fce None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:57.822366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479a06a-19ad-4817-a9e5-9be3223d0476 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:57.947140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a7c9c-3360-44fb-b6c0-76da5cab4fce None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:57.950179 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479a06a-19ad-4817-a9e5-9be3223d0476 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:57.988677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e059ee-8757-41a1-9dcc-64ff309a59dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:57.988913 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0932b-636a-4544-ae2a-49e2daf56bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:57.989340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0932b-636a-4544-ae2a-49e2daf56bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:57.989448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e059ee-8757-41a1-9dcc-64ff309a59dc None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:57.989620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0932b-636a-4544-ae2a-49e2daf56bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:57.989715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e059ee-8757-41a1-9dcc-64ff309a59dc None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:57.989882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0932b-636a-4544-ae2a-49e2daf56bb0 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:57.989974 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e059ee-8757-41a1-9dcc-64ff309a59dc None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:57.990501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0932b-636a-4544-ae2a-49e2daf56bb0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:57.990635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e059ee-8757-41a1-9dcc-64ff309a59dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:57.990790 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0932b-636a-4544-ae2a-49e2daf56bb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:57.990891 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e059ee-8757-41a1-9dcc-64ff309a59dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:57.992153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0932b-636a-4544-ae2a-49e2daf56bb0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:57.992265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e059ee-8757-41a1-9dcc-64ff309a59dc None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:57.998550 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 353/705] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Sep 28 12:09:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:09:58.006922 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 353/706] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:58.012538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0932b-636a-4544-ae2a-49e2daf56bb0 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:58.031992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:58.137607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0932b-636a-4544-ae2a-49e2daf56bb0 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:58.178753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8096b-89aa-4ff8-b203-2e323b73ff36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:58.179403 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8096b-89aa-4ff8-b203-2e323b73ff36 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:58.179681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8096b-89aa-4ff8-b203-2e323b73ff36 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:58.179953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8096b-89aa-4ff8-b203-2e323b73ff36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:58.188098 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8096b-89aa-4ff8-b203-2e323b73ff36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:58.189591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8096b-89aa-4ff8-b203-2e323b73ff36 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:58.196071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29fff3-a614-4973-b8db-169ac278449b None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:58.196381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29fff3-a614-4973-b8db-169ac278449b None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:58.196621 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29fff3-a614-4973-b8db-169ac278449b None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:58.198155 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 354/707] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:58.200774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e1bbf-b602-45b4-93c9-52fe2eef13d0 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:58.201054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e1bbf-b602-45b4-93c9-52fe2eef13d0 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:58.201506 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e1bbf-b602-45b4-93c9-52fe2eef13d0 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:58.205494 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 355/708] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:09:58.205913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae55ba4-9234-4d9f-b111-0e6a20356c61 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:58.206194 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae55ba4-9234-4d9f-b111-0e6a20356c61 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:58.206467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae55ba4-9234-4d9f-b111-0e6a20356c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:58.206649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-4ef8096b-89aa-4ff8-b203-2e323b73ff36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:58.206649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:58.206649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:58.206649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:58.206649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:58.206649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:58.206649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:58.206649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:58.206649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:58.206649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:58.206649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:58.206649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:58.206649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:09:58.206649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:09:58.206649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:09:58.207728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:09:58.207728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:58.207728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:58.207728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:58.207728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:58.207728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:58.207728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:58.207728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:58.207728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:58.207728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:58.207728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:58.207728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:58.207728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:58.207728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:58.207728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:58.207728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:58.207728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:58.208625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:09:58.208625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:09:58.208625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:09:58.208625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:09:58.208625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:09:58.208625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:09:58.208625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:09:58.208625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:58.208625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 354/709] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:09:58.214012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede6bf3-90a1-4a8d-a049-35466b5dc8e6 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:58.214382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede6bf3-90a1-4a8d-a049-35466b5dc8e6 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:58.214655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede6bf3-90a1-4a8d-a049-35466b5dc8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:58.230341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-fae55ba4-9234-4d9f-b111-0e6a20356c61 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:58.231526 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-fae55ba4-9234-4d9f-b111-0e6a20356c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:58.235226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-6ede6bf3-90a1-4a8d-a049-35466b5dc8e6 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:09:58.237639 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-6ede6bf3-90a1-4a8d-a049-35466b5dc8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:09:58.248101 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:58.313687 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 356/710] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:09:58.314838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae55ba4-9234-4d9f-b111-0e6a20356c61 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:58.325283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 355/711] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:09:58.437109 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae55ba4-9234-4d9f-b111-0e6a20356c61 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:58.487477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac67e8-b45e-4286-9b75-d4a99476f82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:58.488091 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac67e8-b45e-4286-9b75-d4a99476f82d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:58.488367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac67e8-b45e-4286-9b75-d4a99476f82d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:58.488637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac67e8-b45e-4286-9b75-d4a99476f82d None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:58.489346 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac67e8-b45e-4286-9b75-d4a99476f82d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:58.489653 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac67e8-b45e-4286-9b75-d4a99476f82d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:58.491079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac67e8-b45e-4286-9b75-d4a99476f82d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:58.504019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 357/712] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:09:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:58.509472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede6bf3-90a1-4a8d-a049-35466b5dc8e6 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:58.517860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:58.604003 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac67e8-b45e-4286-9b75-d4a99476f82d None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:58.616111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:58.633282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede6bf3-90a1-4a8d-a049-35466b5dc8e6 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:58.683096 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07e513-dff5-49b7-960b-39fdd4f6c86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:58.683829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07e513-dff5-49b7-960b-39fdd4f6c86e None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:58.684107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07e513-dff5-49b7-960b-39fdd4f6c86e None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:58.684377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07e513-dff5-49b7-960b-39fdd4f6c86e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:58.685095 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07e513-dff5-49b7-960b-39fdd4f6c86e None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=982b82e8ac424318a934f9149fb91706) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:58.685430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07e513-dff5-49b7-960b-39fdd4f6c86e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:58.686849 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07e513-dff5-49b7-960b-39fdd4f6c86e None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:58.700148 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 356/713] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:09:58 2022] GET /identity/v3/role_assignments?user.id=982b82e8ac424318a934f9149fb91706&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:58.701170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07e513-dff5-49b7-960b-39fdd4f6c86e None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:58.753973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac67e8-b45e-4286-9b75-d4a99476f82d None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:58.801120 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2ef84-0aba-4681-8c69-955773c0e2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:58.801802 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2ef84-0aba-4681-8c69-955773c0e2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:58.805518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2ef84-0aba-4681-8c69-955773c0e2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:58.805811 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2ef84-0aba-4681-8c69-955773c0e2f7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:58.806433 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2ef84-0aba-4681-8c69-955773c0e2f7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:58.807872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2ef84-0aba-4681-8c69-955773c0e2f7 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:58.822502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-a8c2ef84-0aba-4681-8c69-955773c0e2f7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 12:09:58.822502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:58.822502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:09:58.822502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:09:58.822502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:09:58.822502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:09:58.822502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:09:58.822502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:09:58.822502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 12:09:58.822502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:58.822502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:09:58.822502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:58.822502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:58.822502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:58.822502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:58.822502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:58.822502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:58.823622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:58.823622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:58.823622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:58.823622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:58.823622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:58.823622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:58.823622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:09:58.823622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:09:58.823622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:09:58.823622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:09:58.823622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:58.823622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:58.823622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:58.823622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:58.823622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:58.823622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:58.824459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:58.824459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:58.824459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:58.824459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:58.824459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:58.824459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:58.824459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:58.824459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:58.824459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:58.824459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:58.824459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:09:58.824459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:09:58.824459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 12:09:58.824459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:09:58.827772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 358/714] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:09:58 2022] GET /identity/v3/services/network => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:58.832722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2ef84-0aba-4681-8c69-955773c0e2f7 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:58.861967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:58.933569 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:58.959040 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07e513-dff5-49b7-960b-39fdd4f6c86e None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:58.960574 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2ef84-0aba-4681-8c69-955773c0e2f7 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:59.013063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cc863-7f20-4b83-b701-539f473b7e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:59.013786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cc863-7f20-4b83-b701-539f473b7e64 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:59.014073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cc863-7f20-4b83-b701-539f473b7e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:59.016247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cc863-7f20-4b83-b701-539f473b7e64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:59.017058 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296ad9a-d367-4490-af12-25c0bf983344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:59.017705 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296ad9a-d367-4490-af12-25c0bf983344 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:59.017987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296ad9a-d367-4490-af12-25c0bf983344 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:59.018259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296ad9a-d367-4490-af12-25c0bf983344 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:59.018899 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4296ad9a-d367-4490-af12-25c0bf983344 None admin] RBAC: Adding query filter params (name=network) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:59.019200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4296ad9a-d367-4490-af12-25c0bf983344 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:59.020572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4296ad9a-d367-4490-af12-25c0bf983344 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:59.033199 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 359/715] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 12:09:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:59.039651 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1cc863-7f20-4b83-b701-539f473b7e64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:59.041426 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1cc863-7f20-4b83-b701-539f473b7e64 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:59.042317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296ad9a-d367-4490-af12-25c0bf983344 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:59.057802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-4c1cc863-7f20-4b83-b701-539f473b7e64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:59.057802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:09:59.057802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:09:59.057802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:09:59.057802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:09:59.057802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:09:59.057802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:09:59.057802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:09:59.057802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:09:59.057802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:09:59.057802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:09:59.057802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:09:59.057802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:09:59.057802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:09:59.057802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:09:59.057802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:09:59.058813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:09:59.058813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:09:59.058813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:09:59.058813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:09:59.058813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:09:59.058813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:09:59.058813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:09:59.058813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:09:59.058813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:09:59.058813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:09:59.058813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:09:59.058813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:09:59.058813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:09:59.058813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:09:59.058813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:09:59.058813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:09:59.058813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:09:59.059738 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:09:59.059738 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:09:59.059738 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:09:59.062018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 357/716] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:09:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:09:59.073792 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cc863-7f20-4b83-b701-539f473b7e64 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:59.219539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296ad9a-d367-4490-af12-25c0bf983344 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:59.221719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cc863-7f20-4b83-b701-539f473b7e64 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:59.247060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:59.258726 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b173254-3ef2-4d07-ad18-e3468c750ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:59.259699 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b173254-3ef2-4d07-ad18-e3468c750ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:59.260184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b173254-3ef2-4d07-ad18-e3468c750ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:59.260866 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b173254-3ef2-4d07-ad18-e3468c750ed5 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:59.261754 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b173254-3ef2-4d07-ad18-e3468c750ed5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:59.262319 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b173254-3ef2-4d07-ad18-e3468c750ed5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:59.264071 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b173254-3ef2-4d07-ad18-e3468c750ed5 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:59.282297 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 358/717] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:59.283957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b173254-3ef2-4d07-ad18-e3468c750ed5 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:09:59.326545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc8b29-197c-4c36-883b-c9da21508954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:59.327188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc8b29-197c-4c36-883b-c9da21508954 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:59.327467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc8b29-197c-4c36-883b-c9da21508954 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:59.327737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc8b29-197c-4c36-883b-c9da21508954 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:59.329019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc8b29-197c-4c36-883b-c9da21508954 None admin] RBAC: Adding query filter params (type=network) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:09:59.329343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc8b29-197c-4c36-883b-c9da21508954 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:59.330954 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc8b29-197c-4c36-883b-c9da21508954 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:59.339435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:09:59.351174 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 360/718] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 12:09:59 2022] GET /identity/v3/services?type=network => generated 124 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:09:59.397841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b173254-3ef2-4d07-ad18-e3468c750ed5 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:09:59.434510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4cd73-04eb-4fb8-a3dd-28f4ec189f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:09:59.435282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4cd73-04eb-4fb8-a3dd-28f4ec189f20 None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:09:59.435664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4cd73-04eb-4fb8-a3dd-28f4ec189f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:09:59.436058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4cd73-04eb-4fb8-a3dd-28f4ec189f20 None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:09:59.436804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c4cd73-04eb-4fb8-a3dd-28f4ec189f20 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:09:59.439188 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c4cd73-04eb-4fb8-a3dd-28f4ec189f20 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:09:59.469237 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 359/719] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:09:59 2022] POST /identity/v3/users => generated 254 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:10:00.453634 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7a207-9ad0-41c0-b708-34f1518c3937 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:00.454154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7a207-9ad0-41c0-b708-34f1518c3937 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:00.454231 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7a207-9ad0-41c0-b708-34f1518c3937 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:00.455905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 361/720] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:00.461993 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38171488-4fe2-4756-bddc-b7ba95a311b0 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:00.462537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38171488-4fe2-4756-bddc-b7ba95a311b0 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:00.462537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38171488-4fe2-4756-bddc-b7ba95a311b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:00.483052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-38171488-4fe2-4756-bddc-b7ba95a311b0 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:00.484253 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-38171488-4fe2-4756-bddc-b7ba95a311b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:00.515144 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:00.561253 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 360/721] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:10:00.695887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271a217-4bbb-45ea-9a27-6a2ddf697e85 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:00.696220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271a217-4bbb-45ea-9a27-6a2ddf697e85 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:00.696442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271a217-4bbb-45ea-9a27-6a2ddf697e85 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:00.698084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 362/722] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:00.703564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8416b-52cf-49f7-8653-fb67bbceeb02 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:00.703927 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8416b-52cf-49f7-8653-fb67bbceeb02 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:00.704151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8416b-52cf-49f7-8653-fb67bbceeb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:00.725718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-47e8416b-52cf-49f7-8653-fb67bbceeb02 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:00.726914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-47e8416b-52cf-49f7-8653-fb67bbceeb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:00.803979 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 361/723] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:10:00.844844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271a217-4bbb-45ea-9a27-6a2ddf697e85 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:00.966429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271a217-4bbb-45ea-9a27-6a2ddf697e85 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:01.012813 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7cc03-ba58-4414-b6a2-85d4c97d1215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:01.013462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7cc03-ba58-4414-b6a2-85d4c97d1215 None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:01.013748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7cc03-ba58-4414-b6a2-85d4c97d1215 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:01.014023 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7cc03-ba58-4414-b6a2-85d4c97d1215 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:01.014685 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc7cc03-ba58-4414-b6a2-85d4c97d1215 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:01.016302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc7cc03-ba58-4414-b6a2-85d4c97d1215 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:01.037470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 363/724] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 12:10:00 2022] POST /identity/v3/services => generated 251 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:10:01.117518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8416b-52cf-49f7-8653-fb67bbceeb02 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:01.251167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:01.257372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8416b-52cf-49f7-8653-fb67bbceeb02 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:01.322770 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4bf50-e1f1-4013-8848-48913f08f07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:01.323431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4bf50-e1f1-4013-8848-48913f08f07a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:01.323711 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4bf50-e1f1-4013-8848-48913f08f07a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:01.323983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4bf50-e1f1-4013-8848-48913f08f07a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:01.344905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4bf50-e1f1-4013-8848-48913f08f07a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:10:01.349815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-53e4bf50-e1f1-4013-8848-48913f08f07a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:10:01.349815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:01.349815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:01.349815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:01.349815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:01.349815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:01.349815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:01.349815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:01.349815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:01.349815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:01.349815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:01.349815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:01.349815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:10:01.349815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:10:01.349815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:10:01.349815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:10:01.350742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:10:01.350742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:10:01.350742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:10:01.350742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:10:01.350742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:01.350742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:01.350742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:01.350742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:01.350742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:01.350742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:01.350742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:01.350742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:01.350742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:01.350742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:01.350742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:01.350742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:01.351586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:01.351586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:01.351586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:01.351586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:01.351586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:10:01.351586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:10:01.351586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:10:01.351586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:10:01.351586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:10:01.351586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:10:01.351586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:10:01.351586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:01.351586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 362/725] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:10:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:01.356771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7cc03-ba58-4414-b6a2-85d4c97d1215 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:01.415459 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b883889-6363-48ec-9a99-3a786b30a00b None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:01.416083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b883889-6363-48ec-9a99-3a786b30a00b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:01.417022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b883889-6363-48ec-9a99-3a786b30a00b None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:01.418925 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 363/726] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:01.424171 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e0897-f0bb-4a14-92e3-a277c3733111 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:01.424716 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:01.424828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e0897-f0bb-4a14-92e3-a277c3733111 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:01.425058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e0897-f0bb-4a14-92e3-a277c3733111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:01.452237 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-eb3e0897-f0bb-4a14-92e3-a277c3733111 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:01.455028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-eb3e0897-f0bb-4a14-92e3-a277c3733111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:01.471005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7cc03-ba58-4414-b6a2-85d4c97d1215 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:01.510841 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9d70e-e1a8-4c5a-bc10-8237ff648d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:01.511796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9d70e-e1a8-4c5a-bc10-8237ff648d81 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:01.512379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9d70e-e1a8-4c5a-bc10-8237ff648d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:01.512862 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9d70e-e1a8-4c5a-bc10-8237ff648d81 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:01.513801 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9d70e-e1a8-4c5a-bc10-8237ff648d81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:01.514309 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9d70e-e1a8-4c5a-bc10-8237ff648d81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:01.515947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9d70e-e1a8-4c5a-bc10-8237ff648d81 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:01.525882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 364/727] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:10:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:01.532584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9d70e-e1a8-4c5a-bc10-8237ff648d81 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:01.533474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 364/728] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 12:10:01.673245 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9d70e-e1a8-4c5a-bc10-8237ff648d81 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:01.702311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:01.717982 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e066b9-c23e-4f37-b1f2-39403b2cdc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:01.718627 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e066b9-c23e-4f37-b1f2-39403b2cdc69 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:01.718944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e066b9-c23e-4f37-b1f2-39403b2cdc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:01.719230 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e066b9-c23e-4f37-b1f2-39403b2cdc69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:01.734716 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e066b9-c23e-4f37-b1f2-39403b2cdc69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:01.736442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e066b9-c23e-4f37-b1f2-39403b2cdc69 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:01.755435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-48e066b9-c23e-4f37-b1f2-39403b2cdc69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:01.755435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:01.755435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:01.755435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:01.755435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:01.755435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:01.755435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:01.755435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:01.755435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:01.755435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:01.755435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:01.755435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:01.755435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:01.755435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:01.755435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:01.755435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:01.756449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:01.756449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:01.756449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:01.756449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:01.756449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:01.756449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:01.756449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:01.756449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:01.756449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:01.756449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:01.756449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:01.756449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:01.756449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:01.756449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:01.756449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:01.756449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:01.756449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:01.757956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:01.757956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:01.757956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:01.757956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 365/729] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:01.762163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e0897-f0bb-4a14-92e3-a277c3733111 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:01.847036 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e066b9-c23e-4f37-b1f2-39403b2cdc69 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:01.901547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:01.941020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e0897-f0bb-4a14-92e3-a277c3733111 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:02.016335 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e066b9-c23e-4f37-b1f2-39403b2cdc69 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:02.019589 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b748ea6-2a26-4ef8-ad2a-cf2fadc1742d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:02.021237 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b748ea6-2a26-4ef8-ad2a-cf2fadc1742d None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:02.021537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b748ea6-2a26-4ef8-ad2a-cf2fadc1742d None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:02.021818 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b748ea6-2a26-4ef8-ad2a-cf2fadc1742d None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:02.022515 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b748ea6-2a26-4ef8-ad2a-cf2fadc1742d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:02.022825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b748ea6-2a26-4ef8-ad2a-cf2fadc1742d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:02.024266 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b748ea6-2a26-4ef8-ad2a-cf2fadc1742d None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:02.048241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 365/730] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:02.057188 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b748ea6-2a26-4ef8-ad2a-cf2fadc1742d None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:02.085189 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe22b9-320b-4345-b580-0a461a27198a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:02.085829 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe22b9-320b-4345-b580-0a461a27198a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:02.086106 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe22b9-320b-4345-b580-0a461a27198a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:02.086375 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe22b9-320b-4345-b580-0a461a27198a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:02.108808 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe22b9-320b-4345-b580-0a461a27198a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:10:02.116334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-21fe22b9-320b-4345-b580-0a461a27198a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:10:02.116334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:02.116334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:02.116334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:02.116334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:02.116334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:02.116334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:02.116334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:02.116334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:02.116334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:02.116334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:02.116334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:02.116334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:10:02.116334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:10:02.116334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:10:02.116334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:10:02.117311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:10:02.117311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:10:02.117311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:10:02.117311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:10:02.117311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:02.117311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:02.117311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:02.117311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:02.117311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:02.117311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:02.117311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:02.117311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:02.117311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:02.117311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:02.117311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:02.117311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:02.118171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:02.118171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:02.118171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:02.118171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:02.118171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:10:02.118171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:10:02.118171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:10:02.118171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:10:02.118171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:10:02.118171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:10:02.118171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:10:02.118171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:02.118171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 366/731] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:10:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:02.124529 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe22b9-320b-4345-b580-0a461a27198a None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:02.176613 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:02.209462 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b748ea6-2a26-4ef8-ad2a-cf2fadc1742d None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:02.245447 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b3830-5461-4d50-828f-c7aa8718ccde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:02.246080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b3830-5461-4d50-828f-c7aa8718ccde None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:02.246355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b3830-5461-4d50-828f-c7aa8718ccde None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:02.246624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b3830-5461-4d50-828f-c7aa8718ccde None admin] PATH_INFO: `/v3/users/swift` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:02.255167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b3830-5461-4d50-828f-c7aa8718ccde None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:02.256604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b3830-5461-4d50-828f-c7aa8718ccde None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:02.268897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-572b3830-5461-4d50-828f-c7aa8718ccde None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 12:10:02.268897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:02.268897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:02.268897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:02.268897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:02.268897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:02.268897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:02.268897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:02.268897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:02.268897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:02.268897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:02.268897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:02.268897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:10:02.268897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:10:02.268897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:10:02.268897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:02.269911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:02.270900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:02.270900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:02.270900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:10:02.270900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:10:02.270900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:10:02.270900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:10:02.270900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:10:02.270900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:10:02.270900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 12:10:02.270900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:02.270900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 366/732] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:10:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:02.271490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fbdb4-3da8-4183-9ad4-9311d852f62b None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:02.272052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fbdb4-3da8-4183-9ad4-9311d852f62b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:02.272052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774fbdb4-3da8-4183-9ad4-9311d852f62b None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:02.272647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe22b9-320b-4345-b580-0a461a27198a None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:02.277938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 367/733] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:02.282326 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fbdb4-3da8-4183-9ad4-9311d852f62b None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:02.284176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:02.326399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b20e2c-c554-4bbd-9f7a-f3d70415b660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:02.327779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b20e2c-c554-4bbd-9f7a-f3d70415b660 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:02.328804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b20e2c-c554-4bbd-9f7a-f3d70415b660 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:02.329725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b20e2c-c554-4bbd-9f7a-f3d70415b660 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:02.330891 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b20e2c-c554-4bbd-9f7a-f3d70415b660 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:02.331371 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b20e2c-c554-4bbd-9f7a-f3d70415b660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:02.333686 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b20e2c-c554-4bbd-9f7a-f3d70415b660 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:02.349187 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 367/734] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:10:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:02.349187 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee3544-e0b5-4dae-8962-21b3a01776ed None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:02.349187 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee3544-e0b5-4dae-8962-21b3a01776ed None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:02.349187 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee3544-e0b5-4dae-8962-21b3a01776ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:02.369909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-c6ee3544-e0b5-4dae-8962-21b3a01776ed None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:02.371086 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c6ee3544-e0b5-4dae-8962-21b3a01776ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:02.409711 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774fbdb4-3da8-4183-9ad4-9311d852f62b None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:02.445778 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ef51f-df18-4d83-b68d-38a59ae74d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:02.446444 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ef51f-df18-4d83-b68d-38a59ae74d55 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:02.446674 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ef51f-df18-4d83-b68d-38a59ae74d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:02.446944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ef51f-df18-4d83-b68d-38a59ae74d55 None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:02.447693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ef51f-df18-4d83-b68d-38a59ae74d55 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:02.447997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ef51f-df18-4d83-b68d-38a59ae74d55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:02.449915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ef51f-df18-4d83-b68d-38a59ae74d55 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:02.466936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 368/735] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:10:02.466936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee3544-e0b5-4dae-8962-21b3a01776ed None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:02.487423 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 368/736] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Sep 28 12:10:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:02.506164 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ef51f-df18-4d83-b68d-38a59ae74d55 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:02.516914 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:02.603162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:02.630408 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee3544-e0b5-4dae-8962-21b3a01776ed None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:02.654809 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ef51f-df18-4d83-b68d-38a59ae74d55 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:02.669100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff2216-acb0-4213-9645-cc3672539bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:02.669891 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff2216-acb0-4213-9645-cc3672539bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:02.670360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff2216-acb0-4213-9645-cc3672539bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:02.670670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff2216-acb0-4213-9645-cc3672539bc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:02.682371 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff2216-acb0-4213-9645-cc3672539bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:02.683965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff2216-acb0-4213-9645-cc3672539bc5 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:02.697531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a6867-bd70-4d73-95f6-3f2b7c83b369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:02.698340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a6867-bd70-4d73-95f6-3f2b7c83b369 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:02.698791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a6867-bd70-4d73-95f6-3f2b7c83b369 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:02.699201 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a6867-bd70-4d73-95f6-3f2b7c83b369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:02.700203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-2eff2216-acb0-4213-9645-cc3672539bc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:02.700203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:02.700203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:02.700203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:02.700203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:02.700203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:02.700203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:02.700203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:02.700203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:02.700203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:02.700203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:02.700203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:02.700203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:02.700203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:02.700203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:02.700203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:02.701403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:02.701403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:02.701403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:02.701403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:02.701403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:02.701403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:02.701403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:02.701403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:02.701403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:02.701403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:02.701403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:02.701403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:02.701403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:02.701403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:02.701403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:02.701403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:02.701403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:02.702476 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:02.702476 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:02.702476 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:02.704199 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 369/737] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:02.709434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caee36-c22a-441f-b7ce-4a7ba40a294c None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:02.709434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caee36-c22a-441f-b7ce-4a7ba40a294c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:02.709434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96caee36-c22a-441f-b7ce-4a7ba40a294c None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:02.709434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 370/738] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:02.714276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96caee36-c22a-441f-b7ce-4a7ba40a294c None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:02.791962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a6867-bd70-4d73-95f6-3f2b7c83b369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:02.794057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a6867-bd70-4d73-95f6-3f2b7c83b369 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:02.801927 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:02.825507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96caee36-c22a-441f-b7ce-4a7ba40a294c None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:02.858044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:02.861719 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90c576-1ca8-485b-a356-b13d48284c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:02.862326 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90c576-1ca8-485b-a356-b13d48284c02 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:02.862603 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90c576-1ca8-485b-a356-b13d48284c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:02.862884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90c576-1ca8-485b-a356-b13d48284c02 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:02.863500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90c576-1ca8-485b-a356-b13d48284c02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:02.863878 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90c576-1ca8-485b-a356-b13d48284c02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:02.865163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec90c576-1ca8-485b-a356-b13d48284c02 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:02.877411 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 371/739] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:02.882913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd094a-4e27-460b-9768-773e28e19500 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:02.883577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd094a-4e27-460b-9768-773e28e19500 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:02.883863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd094a-4e27-460b-9768-773e28e19500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:02.902137 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-f2fd094a-4e27-460b-9768-773e28e19500 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:02.907910 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 369/740] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:10:02 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 12:10:02.908039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f2fd094a-4e27-460b-9768-773e28e19500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:02.908118 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a6867-bd70-4d73-95f6-3f2b7c83b369 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:02.972922 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 372/741] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 12:10:02.974136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd094a-4e27-460b-9768-773e28e19500 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:03.031506 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a6867-bd70-4d73-95f6-3f2b7c83b369 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:03.066707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a5122-fcd4-4d11-9a00-7be940b89536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:03.067402 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a5122-fcd4-4d11-9a00-7be940b89536 None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:03.067743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a5122-fcd4-4d11-9a00-7be940b89536 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:03.068101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a5122-fcd4-4d11-9a00-7be940b89536 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:03.068806 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a5122-fcd4-4d11-9a00-7be940b89536 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:03.070283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a5122-fcd4-4d11-9a00-7be940b89536 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:03.070758 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:03.091936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 370/742] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 12:10:02 2022] POST /identity/v3/services => generated 240 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:10:03.101413 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a5122-fcd4-4d11-9a00-7be940b89536 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:03.112073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd094a-4e27-460b-9768-773e28e19500 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:03.145162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:03.168196 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e54e5-380b-4939-addd-e84ef991c8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:03.168842 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e54e5-380b-4939-addd-e84ef991c8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:03.169123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e54e5-380b-4939-addd-e84ef991c8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:03.169417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e54e5-380b-4939-addd-e84ef991c8b7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:03.179284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e54e5-380b-4939-addd-e84ef991c8b7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:03.180779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e54e5-380b-4939-addd-e84ef991c8b7 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:03.196473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-653e54e5-380b-4939-addd-e84ef991c8b7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:10:03.196473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:03.196473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:03.196473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:03.196473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:03.196473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:03.196473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:03.196473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:03.196473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:03.196473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:03.196473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:03.196473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:03.196473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:10:03.196473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:10:03.196473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:10:03.196473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:03.197502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:03.198435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:03.198435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:03.198435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:10:03.198435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:10:03.198435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:10:03.198435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:10:03.198435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:10:03.198435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:10:03.198435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:10:03.198435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:03.199660 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 373/743] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 12:10:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:03.204188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e54e5-380b-4939-addd-e84ef991c8b7 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:03.298730 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a5122-fcd4-4d11-9a00-7be940b89536 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:03.337430 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e54e5-380b-4939-addd-e84ef991c8b7 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:03.341408 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc62572-64e5-4644-be2f-53bc4985a133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:03.342429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc62572-64e5-4644-be2f-53bc4985a133 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:03.342890 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc62572-64e5-4644-be2f-53bc4985a133 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:03.343340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc62572-64e5-4644-be2f-53bc4985a133 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:03.356943 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc62572-64e5-4644-be2f-53bc4985a133 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:03.358880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc62572-64e5-4644-be2f-53bc4985a133 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:03.368118 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:03.379013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-4cc62572-64e5-4644-be2f-53bc4985a133 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:03.379013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:03.379013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:03.379013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:03.379013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:03.379013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:03.379013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:03.379013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:03.379013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:03.379013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:03.379013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:03.379013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:03.379013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:03.379013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:03.379013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:03.379013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:03.380004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:03.380004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:03.380004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:03.380004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:03.380004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:03.380004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:03.380004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:03.380004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:03.380004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:03.380004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:03.380004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:03.380004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:03.380004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:03.380004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:03.380004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:03.380004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:03.380004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:03.380895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:03.380895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:03.380895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:03.380895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 371/744] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:03.381215 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc62572-64e5-4644-be2f-53bc4985a133 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:03.383703 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc482e-289a-42e7-9ae9-8a80baad03e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:03.384547 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc482e-289a-42e7-9ae9-8a80baad03e5 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:03.384967 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc482e-289a-42e7-9ae9-8a80baad03e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:03.385444 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc482e-289a-42e7-9ae9-8a80baad03e5 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:03.386316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc482e-289a-42e7-9ae9-8a80baad03e5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:03.386818 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc482e-289a-42e7-9ae9-8a80baad03e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:03.388460 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc482e-289a-42e7-9ae9-8a80baad03e5 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:03.407415 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 374/745] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 12:10:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:03.408673 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc482e-289a-42e7-9ae9-8a80baad03e5 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:03.461620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:03.497516 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc62572-64e5-4644-be2f-53bc4985a133 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:03.522489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc482e-289a-42e7-9ae9-8a80baad03e5 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:03.536543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690631b-5bb8-4967-9b44-48e38f4f4ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:03.537649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690631b-5bb8-4967-9b44-48e38f4f4ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:03.538223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690631b-5bb8-4967-9b44-48e38f4f4ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:03.538689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690631b-5bb8-4967-9b44-48e38f4f4ad4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:03.551236 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690631b-5bb8-4967-9b44-48e38f4f4ad4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:10:03.557347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-2690631b-5bb8-4967-9b44-48e38f4f4ad4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:10:03.557347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:03.557347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:03.557347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:03.557347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:03.557347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:03.557347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:03.557347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:03.557347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:03.557347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:03.557347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:03.557347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:03.557347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:10:03.557347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:10:03.557347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:10:03.557347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:10:03.558282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:10:03.558282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:10:03.558282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:10:03.558282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:10:03.558282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:03.558282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:03.558282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:03.558282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:03.558282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:03.558282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:03.558282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:03.558282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:03.558282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:03.558282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:03.558282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:03.558282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:03.559119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:03.559119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:03.559119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:03.559119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:03.559119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:10:03.559119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:10:03.559119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:10:03.559119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:10:03.559119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:10:03.559119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:10:03.559119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:10:03.559119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:03.561812 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ba360-9f06-419d-87f8-c368c562329c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:03.561962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 372/746] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:10:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:03.563366 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ba360-9f06-419d-87f8-c368c562329c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:03.563830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ba360-9f06-419d-87f8-c368c562329c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:03.563957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690631b-5bb8-4967-9b44-48e38f4f4ad4 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:03.564391 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ba360-9f06-419d-87f8-c368c562329c None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:03.565240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ba360-9f06-419d-87f8-c368c562329c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:03.565721 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ba360-9f06-419d-87f8-c368c562329c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:03.567270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ba360-9f06-419d-87f8-c368c562329c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:03.579833 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 375/747] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:03.580908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ba360-9f06-419d-87f8-c368c562329c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:03.626251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:03.681629 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690631b-5bb8-4967-9b44-48e38f4f4ad4 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:03.690015 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ba360-9f06-419d-87f8-c368c562329c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:03.716685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:03.724124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68efffb-da1a-44bc-825a-91c1eb3ed602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:03.725102 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68efffb-da1a-44bc-825a-91c1eb3ed602 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:03.725655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68efffb-da1a-44bc-825a-91c1eb3ed602 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:03.726074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68efffb-da1a-44bc-825a-91c1eb3ed602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:03.726846 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025588b-cbe7-4af0-ab0e-b1b41005097e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:03.727551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025588b-cbe7-4af0-ab0e-b1b41005097e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:03.727946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025588b-cbe7-4af0-ab0e-b1b41005097e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:03.728319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025588b-cbe7-4af0-ab0e-b1b41005097e None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:03.729081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6025588b-cbe7-4af0-ab0e-b1b41005097e None admin] RBAC: Adding query filter params (name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:03.729509 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6025588b-cbe7-4af0-ab0e-b1b41005097e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:03.730975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6025588b-cbe7-4af0-ab0e-b1b41005097e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:03.741007 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 376/748] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:10:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:03.741424 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68efffb-da1a-44bc-825a-91c1eb3ed602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:03.742142 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025588b-cbe7-4af0-ab0e-b1b41005097e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:03.743362 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68efffb-da1a-44bc-825a-91c1eb3ed602 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:03.760059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-c68efffb-da1a-44bc-825a-91c1eb3ed602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:03.760059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:03.760059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:03.760059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:03.760059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:03.760059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:03.760059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:03.760059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:03.760059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:03.760059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:03.760059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:03.760059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:03.760059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:03.760059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:03.760059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:03.760059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:03.761223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:03.761223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:03.761223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:03.761223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:03.761223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:03.761223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:03.761223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:03.761223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:03.761223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:03.761223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:03.761223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:03.761223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:03.761223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:03.761223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:03.761223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:03.761223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:03.761223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:03.762308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:03.762308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:03.762308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:03.764096 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 373/749] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:03.765268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68efffb-da1a-44bc-825a-91c1eb3ed602 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:03.851377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025588b-cbe7-4af0-ab0e-b1b41005097e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:03.854836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:03.875075 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68efffb-da1a-44bc-825a-91c1eb3ed602 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:03.890391 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4bced-7d2f-47c7-9484-e77a82143d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:03.891111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4bced-7d2f-47c7-9484-e77a82143d8b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:03.891423 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4bced-7d2f-47c7-9484-e77a82143d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:03.891696 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4bced-7d2f-47c7-9484-e77a82143d8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:03.898591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4bced-7d2f-47c7-9484-e77a82143d8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:03.900063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4bced-7d2f-47c7-9484-e77a82143d8b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:03.910531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8ba04-499d-4280-926f-eff570e2dff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:03.911153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8ba04-499d-4280-926f-eff570e2dff8 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:03.911428 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8ba04-499d-4280-926f-eff570e2dff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:03.911696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8ba04-499d-4280-926f-eff570e2dff8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:03.911867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-28d4bced-7d2f-47c7-9484-e77a82143d8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:03.911867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:03.911867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:03.911867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:03.911867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:03.911867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:03.911867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:03.911867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:03.911867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:03.911867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:03.911867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:03.911867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:03.911867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:10:03.911867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:10:03.911867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:10:03.913163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:10:03.913163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:03.913163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:03.913163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:03.913163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:03.913163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:03.913163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:03.913163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:03.913163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:03.913163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:03.913163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:03.913163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:03.913163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:03.913163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:03.913163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:03.913163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:03.913163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:03.914565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:10:03.914565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:10:03.914565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:10:03.914565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:10:03.914565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:10:03.914565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:10:03.914565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:03.914565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:03.914565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 377/750] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:03.914565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4bced-7d2f-47c7-9484-e77a82143d8b None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:03.929419 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8ba04-499d-4280-926f-eff570e2dff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:03.930868 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8ba04-499d-4280-926f-eff570e2dff8 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:03.945518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-91d8ba04-499d-4280-926f-eff570e2dff8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:03.945518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:03.945518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:03.945518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:03.945518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:03.945518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:03.945518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:03.945518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:03.945518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:03.945518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:03.945518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:03.945518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:03.945518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:03.945518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:03.945518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:03.945518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:03.946502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:03.946502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:03.946502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:03.946502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:03.946502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:03.946502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:03.946502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:03.946502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:03.946502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:03.946502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:03.946502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:03.946502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:03.946502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:03.946502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:03.946502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:03.946502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:03.946502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:03.947404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:03.947404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:03.947404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:03.951265 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 374/751] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:03.952484 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44f8af-9c25-491b-ae57-b08a25add80d None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:03.952761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44f8af-9c25-491b-ae57-b08a25add80d None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:03.953112 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44f8af-9c25-491b-ae57-b08a25add80d None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:03.957165 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 375/752] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:03.957165 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44f8af-9c25-491b-ae57-b08a25add80d None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:04.001113 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:04.034191 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4bced-7d2f-47c7-9484-e77a82143d8b None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:04.069004 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1309489-2908-45a8-830a-db94ea52fb7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:04.070029 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1309489-2908-45a8-830a-db94ea52fb7e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:04.070314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1309489-2908-45a8-830a-db94ea52fb7e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:04.070583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1309489-2908-45a8-830a-db94ea52fb7e None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:04.071215 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1309489-2908-45a8-830a-db94ea52fb7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:04.071515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1309489-2908-45a8-830a-db94ea52fb7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:04.072873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1309489-2908-45a8-830a-db94ea52fb7e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:04.072972 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44f8af-9c25-491b-ae57-b08a25add80d None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:04.086136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 378/753] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:04.090021 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1309489-2908-45a8-830a-db94ea52fb7e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:04.110440 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aceac1-98e6-4b66-98b5-edd562fc64c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:04.111302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aceac1-98e6-4b66-98b5-edd562fc64c8 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:04.111578 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aceac1-98e6-4b66-98b5-edd562fc64c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:04.111846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aceac1-98e6-4b66-98b5-edd562fc64c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:04.112526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aceac1-98e6-4b66-98b5-edd562fc64c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:04.112828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aceac1-98e6-4b66-98b5-edd562fc64c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:04.114263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aceac1-98e6-4b66-98b5-edd562fc64c8 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:04.123412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:04.131383 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 376/754] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:10:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:04.131383 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf660b-d322-44d4-a19e-1c56537c32c3 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:04.131383 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf660b-d322-44d4-a19e-1c56537c32c3 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:04.131383 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf660b-d322-44d4-a19e-1c56537c32c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:04.146971 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-0acf660b-d322-44d4-a19e-1c56537c32c3 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:04.148076 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-0acf660b-d322-44d4-a19e-1c56537c32c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:04.221546 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1309489-2908-45a8-830a-db94ea52fb7e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:04.238246 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 377/755] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:04.238246 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf660b-d322-44d4-a19e-1c56537c32c3 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:04.264648 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4a8f8-8f74-4c85-a33d-7637bae082aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:04.265202 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:04.266453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a8f8-8f74-4c85-a33d-7637bae082aa None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:04.266576 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a8f8-8f74-4c85-a33d-7637bae082aa None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:04.266940 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4a8f8-8f74-4c85-a33d-7637bae082aa None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:04.267797 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4a8f8-8f74-4c85-a33d-7637bae082aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:04.268521 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4a8f8-8f74-4c85-a33d-7637bae082aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:04.270820 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4a8f8-8f74-4c85-a33d-7637bae082aa None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:04.288978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 379/756] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:04.289928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4a8f8-8f74-4c85-a33d-7637bae082aa None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:04.353375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf660b-d322-44d4-a19e-1c56537c32c3 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:04.390156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:04.395985 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad3e16-3a24-473f-b113-82c8b3ceb395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:04.397704 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad3e16-3a24-473f-b113-82c8b3ceb395 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:04.398136 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad3e16-3a24-473f-b113-82c8b3ceb395 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:04.398799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad3e16-3a24-473f-b113-82c8b3ceb395 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:04.408586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4a8f8-8f74-4c85-a33d-7637bae082aa None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:04.409379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad3e16-3a24-473f-b113-82c8b3ceb395 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:04.412144 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad3e16-3a24-473f-b113-82c8b3ceb395 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:04.427334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-f9ad3e16-3a24-473f-b113-82c8b3ceb395 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:04.427334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:04.427334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:04.427334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:04.427334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:04.427334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:04.427334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:04.427334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:04.427334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:04.427334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:04.427334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:04.427334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:04.427334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:10:04.427334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:10:04.427334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:10:04.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:10:04.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:04.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:04.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:04.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:04.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:04.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:04.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:04.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:04.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:04.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:04.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:04.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:04.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:04.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:04.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:04.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:04.429679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:10:04.429679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:10:04.429679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:10:04.429679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:10:04.429679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:10:04.429679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:10:04.429679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:04.429679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:04.432141 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 378/757] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:04.433496 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad3e16-3a24-473f-b113-82c8b3ceb395 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:04.448172 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553562cb-3aae-4953-a348-a01662c46bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:04.448919 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553562cb-3aae-4953-a348-a01662c46bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:04.449447 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553562cb-3aae-4953-a348-a01662c46bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:04.449953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553562cb-3aae-4953-a348-a01662c46bc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:04.450639 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553562cb-3aae-4953-a348-a01662c46bc5 None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=975a8d7349994520bcbfb5f75e52aac7) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:04.451198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553562cb-3aae-4953-a348-a01662c46bc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:04.452769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553562cb-3aae-4953-a348-a01662c46bc5 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:04.465420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 380/758] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:10:04 2022] GET /identity/v3/role_assignments?user.id=975a8d7349994520bcbfb5f75e52aac7&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:04.466553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553562cb-3aae-4953-a348-a01662c46bc5 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:04.547303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:04.570530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad3e16-3a24-473f-b113-82c8b3ceb395 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:04.613632 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1637e9-5c42-4033-88c8-0be247e40bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:04.614262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1637e9-5c42-4033-88c8-0be247e40bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:04.614540 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1637e9-5c42-4033-88c8-0be247e40bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:04.614808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1637e9-5c42-4033-88c8-0be247e40bb4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:04.623817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1637e9-5c42-4033-88c8-0be247e40bb4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:04.626253 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1637e9-5c42-4033-88c8-0be247e40bb4 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:04.631043 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553562cb-3aae-4953-a348-a01662c46bc5 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:04.638902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-0d1637e9-5c42-4033-88c8-0be247e40bb4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 12:10:04.638902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:04.638902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:04.638902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:04.638902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:04.638902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:04.638902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:04.638902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:04.638902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:04.638902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:04.638902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:04.638902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:04.638902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:10:04.638902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:10:04.638902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:10:04.638902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:04.640147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:04.641893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:04.641893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:04.641893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:10:04.641893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:10:04.641893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:10:04.641893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:10:04.641893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:10:04.641893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:10:04.641893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 12:10:04.641893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:04.641893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 379/759] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 12:10:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:04.641893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1637e9-5c42-4033-88c8-0be247e40bb4 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:04.689166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03deb0e9-d3a9-4243-bc07-9b6be36f6067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:04.689166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deb0e9-d3a9-4243-bc07-9b6be36f6067 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:04.689166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deb0e9-d3a9-4243-bc07-9b6be36f6067 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:04.689166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03deb0e9-d3a9-4243-bc07-9b6be36f6067 None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:04.689583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03deb0e9-d3a9-4243-bc07-9b6be36f6067 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:04.689583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03deb0e9-d3a9-4243-bc07-9b6be36f6067 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:04.690594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03deb0e9-d3a9-4243-bc07-9b6be36f6067 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:04.709018 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 381/760] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:10:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:04.709964 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03deb0e9-d3a9-4243-bc07-9b6be36f6067 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:04.720773 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:04.759483 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1637e9-5c42-4033-88c8-0be247e40bb4 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:04.801238 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff90fbf-40a1-4f5a-8c71-5f6f3d8e30f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:04.802189 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff90fbf-40a1-4f5a-8c71-5f6f3d8e30f7 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:04.802898 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff90fbf-40a1-4f5a-8c71-5f6f3d8e30f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:04.803374 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff90fbf-40a1-4f5a-8c71-5f6f3d8e30f7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:04.804453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff90fbf-40a1-4f5a-8c71-5f6f3d8e30f7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:04.806034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff90fbf-40a1-4f5a-8c71-5f6f3d8e30f7 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:04.819589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03deb0e9-d3a9-4243-bc07-9b6be36f6067 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:04.820216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-dff90fbf-40a1-4f5a-8c71-5f6f3d8e30f7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 12:10:04.820216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:04.820216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:04.820216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:04.820216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:04.820216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:04.820216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:04.820216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:04.820216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 12:10:04.820216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:04.820216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:04.820216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:04.820216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:04.820216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:04.820216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:04.820216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:04.820216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:04.821205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:04.821205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:04.821205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:04.821205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:04.821205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:04.821205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:04.821205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:04.821205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:04.821205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:04.821205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:04.821205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:04.821205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:04.821205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:04.821205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:04.821205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:04.821205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:04.822052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:04.822052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:04.822052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:04.822052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:04.822052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:04.822052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:04.822052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:04.822052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:04.822052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:04.822052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:04.822052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:04.822052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:04.822052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 12:10:04.822052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:04.825205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 380/761] 10.222.0.60 () {56 vars in 1129 bytes} [Wed Sep 28 12:10:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:04.826184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff90fbf-40a1-4f5a-8c71-5f6f3d8e30f7 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:04.845315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:04.878019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de5f08-7d8e-4d74-809b-630e58dab1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:04.879418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de5f08-7d8e-4d74-809b-630e58dab1ac None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:04.879882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de5f08-7d8e-4d74-809b-630e58dab1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:04.880403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de5f08-7d8e-4d74-809b-630e58dab1ac None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:04.882026 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de5f08-7d8e-4d74-809b-630e58dab1ac None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:04.882539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de5f08-7d8e-4d74-809b-630e58dab1ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:04.884870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de5f08-7d8e-4d74-809b-630e58dab1ac None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:04.910350 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 382/762] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Sep 28 12:10:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:04.912478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de5f08-7d8e-4d74-809b-630e58dab1ac None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:04.947208 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff90fbf-40a1-4f5a-8c71-5f6f3d8e30f7 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:04.986935 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2d5e8-6e13-4d11-a736-e9c1c01d9020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:04.987721 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2d5e8-6e13-4d11-a736-e9c1c01d9020 None admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:04.988038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2d5e8-6e13-4d11-a736-e9c1c01d9020 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:04.988382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2d5e8-6e13-4d11-a736-e9c1c01d9020 None admin] PATH_INFO: `/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/982b82e8ac424318a934f9149fb91706/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:04.990998 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2d5e8-6e13-4d11-a736-e9c1c01d9020 None admin] RBAC: Authorizing `identity:create_grant(project_id=6335d0db1f4844c39c255909189e0ce9, user_id=982b82e8ac424318a934f9149fb91706, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:04.992895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2d5e8-6e13-4d11-a736-e9c1c01d9020 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:05.002237 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:05.017366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 381/763] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:10:04 2022] PUT /identity/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/982b82e8ac424318a934f9149fb91706/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:10:05.018715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2d5e8-6e13-4d11-a736-e9c1c01d9020 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:05.073392 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de5f08-7d8e-4d74-809b-630e58dab1ac None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:05.148310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b91ee-33fc-46ba-bd92-17394b8007b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:05.152987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b91ee-33fc-46ba-bd92-17394b8007b7 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:05.153579 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b91ee-33fc-46ba-bd92-17394b8007b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:05.154187 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b91ee-33fc-46ba-bd92-17394b8007b7 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:05.155299 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b91ee-33fc-46ba-bd92-17394b8007b7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:05.157086 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b91ee-33fc-46ba-bd92-17394b8007b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:05.158824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b91ee-33fc-46ba-bd92-17394b8007b7 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:05.172726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 383/764] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Sep 28 12:10:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:05.178551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b91ee-33fc-46ba-bd92-17394b8007b7 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:05.207658 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:05.250672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2d5e8-6e13-4d11-a736-e9c1c01d9020 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:05.293805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa8e14-7947-4de5-a27f-ef103218eb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:05.294845 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa8e14-7947-4de5-a27f-ef103218eb4b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:05.295321 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa8e14-7947-4de5-a27f-ef103218eb4b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:05.295797 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa8e14-7947-4de5-a27f-ef103218eb4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:05.329900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b91ee-33fc-46ba-bd92-17394b8007b7 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:05.363938 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:05.371797 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f5e7b-2ac9-4cfa-81d0-8f8e4339bba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:05.372442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f5e7b-2ac9-4cfa-81d0-8f8e4339bba7 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:05.372725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f5e7b-2ac9-4cfa-81d0-8f8e4339bba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:05.372997 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f5e7b-2ac9-4cfa-81d0-8f8e4339bba7 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:05.373795 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f5e7b-2ac9-4cfa-81d0-8f8e4339bba7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:05.374104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f5e7b-2ac9-4cfa-81d0-8f8e4339bba7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:05.376092 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f5e7b-2ac9-4cfa-81d0-8f8e4339bba7 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:05.382804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 384/765] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Sep 28 12:10:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:05.383482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa8e14-7947-4de5-a27f-ef103218eb4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:05.389330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa8e14-7947-4de5-a27f-ef103218eb4b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:05.407049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f5e7b-2ac9-4cfa-81d0-8f8e4339bba7 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:05.471058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:05.505370 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 382/766] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:10:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:10:05.507748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba4379-de01-4805-b95c-0b2f1a8f6397 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:05.508231 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba4379-de01-4805-b95c-0b2f1a8f6397 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:05.508700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba4379-de01-4805-b95c-0b2f1a8f6397 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:05.510177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 383/767] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:10:05.511057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba4379-de01-4805-b95c-0b2f1a8f6397 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:05.567875 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f5e7b-2ac9-4cfa-81d0-8f8e4339bba7 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:05.608508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e020b29-91c9-4c10-ba34-535687d302cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:05.609361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e020b29-91c9-4c10-ba34-535687d302cc None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:05.609834 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e020b29-91c9-4c10-ba34-535687d302cc None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:05.610196 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e020b29-91c9-4c10-ba34-535687d302cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:05.611647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e020b29-91c9-4c10-ba34-535687d302cc None admin] RBAC: Adding query filter params (interface=public, service_id=c8349c9b4cb9457bb99dbe9168ba36f9, region_id=RegionOne) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:05.611647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e020b29-91c9-4c10-ba34-535687d302cc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:05.613024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e020b29-91c9-4c10-ba34-535687d302cc None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:05.625849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 385/768] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Sep 28 12:10:05 2022] GET /identity/v3/endpoints?service_id=c8349c9b4cb9457bb99dbe9168ba36f9&interface=public®ion_id=RegionOne => generated 194 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:05.625849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b0fc9-5460-4408-b9bb-705aee5af7de None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:05.625849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b0fc9-5460-4408-b9bb-705aee5af7de None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:05.625849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b0fc9-5460-4408-b9bb-705aee5af7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:05.643688 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-9d8b0fc9-5460-4408-b9bb-705aee5af7de None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:05.644927 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-9d8b0fc9-5460-4408-b9bb-705aee5af7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:05.654369 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba4379-de01-4805-b95c-0b2f1a8f6397 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:05.702376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21710f62-3438-4862-abf6-367ddff07e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:05.704124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21710f62-3438-4862-abf6-367ddff07e62 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:05.705009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21710f62-3438-4862-abf6-367ddff07e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:05.705502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21710f62-3438-4862-abf6-367ddff07e62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:05.708186 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:05.722029 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21710f62-3438-4862-abf6-367ddff07e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:05.724051 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21710f62-3438-4862-abf6-367ddff07e62 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:05.724863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 386/769] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:05.725838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b0fc9-5460-4408-b9bb-705aee5af7de None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:05.744241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-21710f62-3438-4862-abf6-367ddff07e62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:05.744241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:05.744241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:05.744241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:05.744241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:05.744241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:05.744241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:05.744241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:05.744241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:05.744241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:05.744241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:05.744241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:05.744241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:05.744241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:05.744241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:05.744241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:05.745270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:05.745270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:05.745270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:05.745270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:05.745270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:05.745270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:05.745270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:05.745270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:05.745270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:05.745270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:05.745270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:05.745270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:05.745270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:05.745270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:05.745270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:05.745270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:05.745270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:05.746176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:05.746176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:05.746176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:05.746908 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 384/770] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:05.752981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21710f62-3438-4862-abf6-367ddff07e62 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:05.786879 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:05.885523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b0fc9-5460-4408-b9bb-705aee5af7de None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:05.933342 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21710f62-3438-4862-abf6-367ddff07e62 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:05.936606 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cd379-7437-455c-bda8-f7c9fcaa6f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:05.937253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cd379-7437-455c-bda8-f7c9fcaa6f60 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:05.938099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cd379-7437-455c-bda8-f7c9fcaa6f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:05.938592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cd379-7437-455c-bda8-f7c9fcaa6f60 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:05.939261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cd379-7437-455c-bda8-f7c9fcaa6f60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:05.940636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cd379-7437-455c-bda8-f7c9fcaa6f60 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:05.960163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 387/771] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 12:10:05 2022] GET /identity/v3/services => generated 772 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:05.986033 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cf086-1021-4f6f-8f92-9a6de9e21c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:05.986583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cf086-1021-4f6f-8f92-9a6de9e21c15 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:05.986867 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cf086-1021-4f6f-8f92-9a6de9e21c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:05.987137 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cf086-1021-4f6f-8f92-9a6de9e21c15 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:05.987795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1cf086-1021-4f6f-8f92-9a6de9e21c15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:05.988099 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1cf086-1021-4f6f-8f92-9a6de9e21c15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:05.989519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1cf086-1021-4f6f-8f92-9a6de9e21c15 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:06.004189 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 385/772] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:06.010295 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cd379-7437-455c-bda8-f7c9fcaa6f60 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:06.092489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:06.148021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cf086-1021-4f6f-8f92-9a6de9e21c15 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:06.174661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cd379-7437-455c-bda8-f7c9fcaa6f60 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:06.222746 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e698e8-c3b7-453e-ad38-858d0d328fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:06.223905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e698e8-c3b7-453e-ad38-858d0d328fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:06.224319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e698e8-c3b7-453e-ad38-858d0d328fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:06.224595 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e698e8-c3b7-453e-ad38-858d0d328fe4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:06.231749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e698e8-c3b7-453e-ad38-858d0d328fe4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:06.233241 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e698e8-c3b7-453e-ad38-858d0d328fe4 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:06.246512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-b9e698e8-c3b7-453e-ad38-858d0d328fe4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:06.246512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:06.246512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:06.246512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:06.246512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:06.246512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:06.246512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:06.246512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:06.246512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:06.246512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:06.246512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:06.246512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:06.246512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:10:06.246512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:10:06.246512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:10:06.247395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:10:06.247395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:06.247395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:06.247395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:06.247395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:06.247395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:06.247395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:06.247395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:06.247395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:06.247395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:06.247395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:06.247395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:06.247395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:06.247395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:06.247395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:06.247395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:06.247395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:06.248328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:10:06.248328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:10:06.248328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:10:06.248328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:10:06.248328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:10:06.248328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:10:06.248328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:06.248328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:06.248751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:06.248824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 388/773] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:06.253510 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e698e8-c3b7-453e-ad38-858d0d328fe4 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:06.270432 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cf086-1021-4f6f-8f92-9a6de9e21c15 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:06.313026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37005585-7953-4d72-a157-e29c85374051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:06.313813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37005585-7953-4d72-a157-e29c85374051 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:06.314154 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37005585-7953-4d72-a157-e29c85374051 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:06.314495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37005585-7953-4d72-a157-e29c85374051 None admin] PATH_INFO: `/v3/services/network` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:06.315182 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37005585-7953-4d72-a157-e29c85374051 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:06.316713 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37005585-7953-4d72-a157-e29c85374051 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:06.325697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-37005585-7953-4d72-a157-e29c85374051 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 12:10:06.325697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:06.325697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:06.325697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:06.325697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:06.325697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:06.325697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:06.325697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:06.325697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 12:10:06.325697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:06.325697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:06.325697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:06.325697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:06.325697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:06.325697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:06.325697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:06.325697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:06.327227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:06.327227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:06.327227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:06.327227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:06.327227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:06.327227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:06.327227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:06.327227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:06.327227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:06.327227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:06.327227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:06.327227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:06.327227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:06.327227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:06.327227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:06.327227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:06.328821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:06.328821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:06.328821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:06.328821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:06.328821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:06.328821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:06.328821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:06.328821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:06.328821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:06.328821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:06.328821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:06.328821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:06.328821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 12:10:06.328821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:06.330503 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 386/774] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:06 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:06.335791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37005585-7953-4d72-a157-e29c85374051 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:06.403786 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e698e8-c3b7-453e-ad38-858d0d328fe4 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:06.433181 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:06.477771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515df1aa-661b-4583-9651-a4f3e9ec0d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:06.478383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515df1aa-661b-4583-9651-a4f3e9ec0d3d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:06.478663 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515df1aa-661b-4583-9651-a4f3e9ec0d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:06.478934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515df1aa-661b-4583-9651-a4f3e9ec0d3d None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:06.479627 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515df1aa-661b-4583-9651-a4f3e9ec0d3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:06.479933 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515df1aa-661b-4583-9651-a4f3e9ec0d3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:06.481383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515df1aa-661b-4583-9651-a4f3e9ec0d3d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:06.497582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37005585-7953-4d72-a157-e29c85374051 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:06.511810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 389/775] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:10:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:06.517516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515df1aa-661b-4583-9651-a4f3e9ec0d3d None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:06.558080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b39ae-f598-49a6-baf2-7f13aeb03009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:06.558737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b39ae-f598-49a6-baf2-7f13aeb03009 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:06.559019 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b39ae-f598-49a6-baf2-7f13aeb03009 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:06.559287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b39ae-f598-49a6-baf2-7f13aeb03009 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:06.559918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b39ae-f598-49a6-baf2-7f13aeb03009 None admin] RBAC: Adding query filter params (name=network) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:06.560225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b39ae-f598-49a6-baf2-7f13aeb03009 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:06.562709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b39ae-f598-49a6-baf2-7f13aeb03009 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:06.572963 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 387/776] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 12:10:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:06.578561 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b39ae-f598-49a6-baf2-7f13aeb03009 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:06.600192 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:06.684609 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515df1aa-661b-4583-9651-a4f3e9ec0d3d None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:06.751084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2d7ee-caeb-4483-8fd4-c68b415bd62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:06.752237 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2d7ee-caeb-4483-8fd4-c68b415bd62e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:06.753546 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2d7ee-caeb-4483-8fd4-c68b415bd62e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:06.754858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2d7ee-caeb-4483-8fd4-c68b415bd62e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:06.756379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2d7ee-caeb-4483-8fd4-c68b415bd62e None admin] RBAC: Adding query filter params (role.id=60963384f3e54d9cbfbc8d12023a6cd9, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=bc69cf267bea41bf890bc7551d84ad6f) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:06.757319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2d7ee-caeb-4483-8fd4-c68b415bd62e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:06.759761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2d7ee-caeb-4483-8fd4-c68b415bd62e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:06.769565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b39ae-f598-49a6-baf2-7f13aeb03009 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:06.774246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 390/777] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:10:06 2022] GET /identity/v3/role_assignments?user.id=bc69cf267bea41bf890bc7551d84ad6f&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=60963384f3e54d9cbfbc8d12023a6cd9 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:06.811486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22fa1f-556f-4180-9e76-ee324e8919dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:06.813181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22fa1f-556f-4180-9e76-ee324e8919dc None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:06.813808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22fa1f-556f-4180-9e76-ee324e8919dc None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:06.814282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22fa1f-556f-4180-9e76-ee324e8919dc None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:06.815122 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22fa1f-556f-4180-9e76-ee324e8919dc None admin] RBAC: Adding query filter params (type=network) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:06.815623 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22fa1f-556f-4180-9e76-ee324e8919dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:06.817228 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22fa1f-556f-4180-9e76-ee324e8919dc None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:06.827551 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 388/778] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 12:10:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:06.833400 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2d7ee-caeb-4483-8fd4-c68b415bd62e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:06.891220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:06.986888 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2d7ee-caeb-4483-8fd4-c68b415bd62e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:07.023142 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9fe26-0192-4056-bd54-9c3814e32c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:07.024198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9fe26-0192-4056-bd54-9c3814e32c59 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:07.024682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9fe26-0192-4056-bd54-9c3814e32c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:07.025174 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9fe26-0192-4056-bd54-9c3814e32c59 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:07.026078 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9fe26-0192-4056-bd54-9c3814e32c59 None admin] RBAC: Adding query filter params (interface=public, service_id=4fe4dcc912104b99a7ef649c10618201, region_id=RegionOne) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:07.026584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9fe26-0192-4056-bd54-9c3814e32c59 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:07.028162 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9fe26-0192-4056-bd54-9c3814e32c59 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:07.034569 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 391/779] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Sep 28 12:10:06 2022] GET /identity/v3/endpoints?service_id=4fe4dcc912104b99a7ef649c10618201&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:07.040927 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22fa1f-556f-4180-9e76-ee324e8919dc None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:07.116950 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bbb99-df99-4aec-84cc-e62ce311ee0e None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:07.117650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bbb99-df99-4aec-84cc-e62ce311ee0e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:07.118133 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536bbb99-df99-4aec-84cc-e62ce311ee0e None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:07.119694 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 392/780] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:07.138267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318eb0d-098c-4869-898d-981407abea3d None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:07.138597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318eb0d-098c-4869-898d-981407abea3d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:07.138812 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318eb0d-098c-4869-898d-981407abea3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:07.149039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:07.178138 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-4318eb0d-098c-4869-898d-981407abea3d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:07.179338 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-4318eb0d-098c-4869-898d-981407abea3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:07.189542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22fa1f-556f-4180-9e76-ee324e8919dc None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:07.226930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14f237-e2e0-4fa1-b8dc-1af7a57c2a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:07.227931 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14f237-e2e0-4fa1-b8dc-1af7a57c2a8a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:07.228415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14f237-e2e0-4fa1-b8dc-1af7a57c2a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:07.228891 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14f237-e2e0-4fa1-b8dc-1af7a57c2a8a None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:07.229729 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14f237-e2e0-4fa1-b8dc-1af7a57c2a8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:07.231572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14f237-e2e0-4fa1-b8dc-1af7a57c2a8a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:07.238710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 389/781] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 12:10:07 2022] GET /identity/v3/services => generated 772 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:07.289832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 393/782] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:07.625993 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14f237-e2e0-4fa1-b8dc-1af7a57c2a8a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:07.754650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14f237-e2e0-4fa1-b8dc-1af7a57c2a8a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:07.768242 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6307b0-78f6-484f-b0f1-b9c3adbd4fc8 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:07.769188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6307b0-78f6-484f-b0f1-b9c3adbd4fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:07.769697 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6307b0-78f6-484f-b0f1-b9c3adbd4fc8 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:07.771467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 394/783] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:07.777103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de478379-f400-4a4b-a240-2f2a3a452107 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:07.777726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de478379-f400-4a4b-a240-2f2a3a452107 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:07.778224 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de478379-f400-4a4b-a240-2f2a3a452107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:07.800967 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-de478379-f400-4a4b-a240-2f2a3a452107 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:07.803197 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-de478379-f400-4a4b-a240-2f2a3a452107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:07.810974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60980f9c-b027-4602-80c6-630e93991843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:07.811627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60980f9c-b027-4602-80c6-630e93991843 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:07.812251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60980f9c-b027-4602-80c6-630e93991843 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:07.812251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60980f9c-b027-4602-80c6-630e93991843 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:07.835225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:07.837567 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:07.845779 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60980f9c-b027-4602-80c6-630e93991843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:10:07.853922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-60980f9c-b027-4602-80c6-630e93991843 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:10:07.853922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:07.853922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:07.853922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:07.853922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:07.853922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:07.853922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:07.853922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:07.853922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:07.853922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:07.853922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:07.853922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:07.853922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:10:07.853922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:10:07.853922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:10:07.853922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:10:07.854940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:10:07.854940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:10:07.854940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:10:07.854940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:10:07.854940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:07.854940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:07.854940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:07.854940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:07.854940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:07.854940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:07.854940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:07.854940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:07.854940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:07.854940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:07.854940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:07.854940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:07.855799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:07.855799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:07.855799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:07.855799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:07.855799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:10:07.855799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:10:07.855799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:10:07.855799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:10:07.855799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:10:07.855799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:10:07.855799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:10:07.855799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:07.855799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 390/784] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:10:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:07.861759 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60980f9c-b027-4602-80c6-630e93991843 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:07.896182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 395/785] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:08.018163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60980f9c-b027-4602-80c6-630e93991843 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:08.055908 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac69d0-d553-47d0-ae3c-3e646a1be0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:08.056908 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac69d0-d553-47d0-ae3c-3e646a1be0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:08.057413 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac69d0-d553-47d0-ae3c-3e646a1be0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:08.057908 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac69d0-d553-47d0-ae3c-3e646a1be0a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:08.058770 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac69d0-d553-47d0-ae3c-3e646a1be0a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:08.059323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac69d0-d553-47d0-ae3c-3e646a1be0a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:08.060909 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac69d0-d553-47d0-ae3c-3e646a1be0a6 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:08.072556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 391/786] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:10:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:08.078698 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de478379-f400-4a4b-a240-2f2a3a452107 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:08.232479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de478379-f400-4a4b-a240-2f2a3a452107 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:08.271264 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900f507-4f71-467b-97bc-989efadc327e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:08.272278 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900f507-4f71-467b-97bc-989efadc327e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:08.272757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900f507-4f71-467b-97bc-989efadc327e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:08.273250 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900f507-4f71-467b-97bc-989efadc327e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:08.284432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900f507-4f71-467b-97bc-989efadc327e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:08.286180 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900f507-4f71-467b-97bc-989efadc327e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:08.301587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-8900f507-4f71-467b-97bc-989efadc327e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:08.301587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:08.301587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:08.301587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:08.301587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:08.301587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:08.301587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:08.301587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:08.301587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:08.301587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:08.301587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:08.301587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:08.301587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:08.301587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:08.301587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:08.301587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:08.302572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:08.302572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:08.302572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:08.302572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:08.302572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:08.302572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:08.302572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:08.302572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:08.302572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:08.302572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:08.302572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:08.302572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:08.302572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:08.302572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:08.302572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:08.302572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:08.302572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:08.303450 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:08.303450 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:08.303450 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:08.305560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 396/787] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:08.314468 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac69d0-d553-47d0-ae3c-3e646a1be0a6 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:08.390805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:08.444857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac69d0-d553-47d0-ae3c-3e646a1be0a6 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:08.463050 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900f507-4f71-467b-97bc-989efadc327e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:08.488560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:08.518979 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5035fad-e1ed-4b68-8659-cdb39b982c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:08.519700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5035fad-e1ed-4b68-8659-cdb39b982c9f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:08.519984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5035fad-e1ed-4b68-8659-cdb39b982c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:08.520259 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5035fad-e1ed-4b68-8659-cdb39b982c9f None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:08.520905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5035fad-e1ed-4b68-8659-cdb39b982c9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:08.521235 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5035fad-e1ed-4b68-8659-cdb39b982c9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:08.522782 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5035fad-e1ed-4b68-8659-cdb39b982c9f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:08.549842 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 392/788] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:08.555214 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5035fad-e1ed-4b68-8659-cdb39b982c9f None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:08.608387 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900f507-4f71-467b-97bc-989efadc327e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:08.646228 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d56a7-defa-41bb-9f70-a7cdcb5e2136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:08.647248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d56a7-defa-41bb-9f70-a7cdcb5e2136 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:08.647732 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d56a7-defa-41bb-9f70-a7cdcb5e2136 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:08.648323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d56a7-defa-41bb-9f70-a7cdcb5e2136 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:08.660555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d56a7-defa-41bb-9f70-a7cdcb5e2136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:10:08.664595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5035fad-e1ed-4b68-8659-cdb39b982c9f None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:08.668669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-aa6d56a7-defa-41bb-9f70-a7cdcb5e2136 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:10:08.668669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:08.668669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:08.668669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:08.668669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:08.668669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:08.668669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:08.668669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:08.668669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:08.668669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:08.668669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:08.668669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:08.668669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:10:08.668669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:10:08.668669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:10:08.668669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:10:08.669586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:10:08.669586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:10:08.669586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:10:08.669586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:10:08.669586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:08.669586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:08.669586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:08.669586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:08.669586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:08.669586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:08.669586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:08.669586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:08.669586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:08.669586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:08.669586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:08.669586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:08.670448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:08.670448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:08.670448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:08.670448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:08.670448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:10:08.670448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:10:08.670448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:10:08.670448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:10:08.670448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:10:08.670448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:10:08.670448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:10:08.670448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:08.673793 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 397/789] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:10:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:08.678704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f3d96-a836-42a9-9278-2240d363618c None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:08.678704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f3d96-a836-42a9-9278-2240d363618c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:08.678704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f3d96-a836-42a9-9278-2240d363618c None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:08.681212 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 398/790] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:08.681885 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f3d96-a836-42a9-9278-2240d363618c None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:08.700926 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:08.717666 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415a521-0782-468b-a80a-3fd24ac6e5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:08.719984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415a521-0782-468b-a80a-3fd24ac6e5ba None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:08.720794 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415a521-0782-468b-a80a-3fd24ac6e5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:08.721339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415a521-0782-468b-a80a-3fd24ac6e5ba None admin] PATH_INFO: `/v3/users/swift` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:08.730560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415a521-0782-468b-a80a-3fd24ac6e5ba None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:08.732713 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415a521-0782-468b-a80a-3fd24ac6e5ba None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:08.747939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-b415a521-0782-468b-a80a-3fd24ac6e5ba None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 12:10:08.747939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:08.747939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:08.747939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:08.747939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:08.747939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:08.747939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:08.747939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:08.747939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:08.747939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:08.747939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:08.747939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:08.747939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:10:08.747939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:10:08.747939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:10:08.747939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:08.748916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:08.750052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:08.750052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:08.750052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:10:08.750052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:10:08.750052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:10:08.750052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:10:08.750052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:10:08.750052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:10:08.750052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 12:10:08.750052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:08.753324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 393/791] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:10:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:08.754763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0297db4-d6de-4230-a764-e675b38a566f None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:08.755297 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0297db4-d6de-4230-a764-e675b38a566f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:08.755787 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0297db4-d6de-4230-a764-e675b38a566f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:08.769597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:08.777123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-e0297db4-d6de-4230-a764-e675b38a566f None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:08.778665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e0297db4-d6de-4230-a764-e675b38a566f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:08.806237 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f3d96-a836-42a9-9278-2240d363618c None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:08.859760 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9c40e-d565-4819-811a-de0f14f7710c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:08.860373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9c40e-d565-4819-811a-de0f14f7710c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:08.860654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9c40e-d565-4819-811a-de0f14f7710c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:08.861225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9c40e-d565-4819-811a-de0f14f7710c None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:08.861331 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 394/792] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:08.861904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9c40e-d565-4819-811a-de0f14f7710c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:08.862069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0297db4-d6de-4230-a764-e675b38a566f None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:08.862164 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9c40e-d565-4819-811a-de0f14f7710c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:08.863711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9c40e-d565-4819-811a-de0f14f7710c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:08.897605 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 399/793] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:10:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:08.903119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9c40e-d565-4819-811a-de0f14f7710c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:08.973215 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0297db4-d6de-4230-a764-e675b38a566f None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:08.984802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:09.011321 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdce899-685c-4091-b87c-7d4d6b30c303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:09.012286 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdce899-685c-4091-b87c-7d4d6b30c303 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:09.012764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdce899-685c-4091-b87c-7d4d6b30c303 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:09.013302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdce899-685c-4091-b87c-7d4d6b30c303 None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:09.014282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdce899-685c-4091-b87c-7d4d6b30c303 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:09.014786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdce899-685c-4091-b87c-7d4d6b30c303 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:09.017163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdce899-685c-4091-b87c-7d4d6b30c303 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:09.035631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 395/794] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Sep 28 12:10:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:09.045171 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdce899-685c-4091-b87c-7d4d6b30c303 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:09.127861 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9c40e-d565-4819-811a-de0f14f7710c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:09.177241 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f13ef7-e706-447c-8ca5-e7aa59efaa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:09.177881 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13ef7-e706-447c-8ca5-e7aa59efaa78 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:09.178469 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13ef7-e706-447c-8ca5-e7aa59efaa78 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:09.178742 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f13ef7-e706-447c-8ca5-e7aa59efaa78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:09.185928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:09.193065 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f13ef7-e706-447c-8ca5-e7aa59efaa78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:09.194913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f13ef7-e706-447c-8ca5-e7aa59efaa78 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:09.212146 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdce899-685c-4091-b87c-7d4d6b30c303 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:09.214200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-b8f13ef7-e706-447c-8ca5-e7aa59efaa78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:09.214200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:09.214200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:09.214200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:09.214200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:09.214200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:09.214200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:09.214200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:09.214200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:09.214200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:09.214200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:09.214200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:09.214200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:09.214200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:09.214200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:09.214200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:09.215330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:09.215330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:09.215330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:09.215330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:09.215330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:09.215330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:09.215330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:09.215330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:09.215330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:09.215330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:09.215330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:09.215330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:09.215330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:09.215330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:09.215330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:09.215330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:09.215330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:09.216444 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:09.216444 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:09.216444 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:09.221187 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 400/795] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:09.221187 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f13ef7-e706-447c-8ca5-e7aa59efaa78 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:09.261357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fad91-45c9-4b32-a8d6-b2119c4621ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:09.262301 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fad91-45c9-4b32-a8d6-b2119c4621ee None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:09.262729 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fad91-45c9-4b32-a8d6-b2119c4621ee None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:09.263267 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fad91-45c9-4b32-a8d6-b2119c4621ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:09.276474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25fad91-45c9-4b32-a8d6-b2119c4621ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:09.278385 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25fad91-45c9-4b32-a8d6-b2119c4621ee None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:09.300969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-c25fad91-45c9-4b32-a8d6-b2119c4621ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:09.300969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:09.300969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:09.300969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:09.300969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:09.300969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:09.300969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:09.300969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:09.300969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:09.300969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:09.300969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:09.300969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:09.300969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:09.300969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:09.300969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:09.300969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:09.302178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:09.302178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:09.302178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:09.302178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:09.302178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:09.302178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:09.302178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:09.302178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:09.302178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:09.302178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:09.302178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:09.302178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:09.302178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:09.302178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:09.302178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:09.302178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:09.302178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:09.303078 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:09.303078 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:09.303078 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:09.303266 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 396/796] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:09.305183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fad91-45c9-4b32-a8d6-b2119c4621ee None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:09.342811 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:09.353931 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f13ef7-e706-447c-8ca5-e7aa59efaa78 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:09.392578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fef7e-dc03-4004-8d3b-690d69f07b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:09.393394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fef7e-dc03-4004-8d3b-690d69f07b2a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:09.393677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fef7e-dc03-4004-8d3b-690d69f07b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:09.393949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fef7e-dc03-4004-8d3b-690d69f07b2a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:09.394523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267fef7e-dc03-4004-8d3b-690d69f07b2a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:09.395881 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267fef7e-dc03-4004-8d3b-690d69f07b2a None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:09.404731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-267fef7e-dc03-4004-8d3b-690d69f07b2a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 12:10:09.404731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:09.404731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:09.404731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:09.404731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:09.404731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:09.404731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:09.404731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:09.404731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 12:10:09.404731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:09.404731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:09.404731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:09.404731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:09.404731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:09.404731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:09.404731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:09.404731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:09.405737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:09.405737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:09.405737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:09.405737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:09.405737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:09.405737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:09.405737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:09.405737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:09.405737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:09.405737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:09.405737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:09.405737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:09.405737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:09.405737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:09.405737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:09.405737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:09.407141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:09.407141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:09.407141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:09.407141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:09.407141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:09.407141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:09.407141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:09.407141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:09.407141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:09.407141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:09.407141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:09.407141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:09.407141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 12:10:09.407141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:09.407141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 401/797] 10.222.0.60 () {56 vars in 1129 bytes} [Wed Sep 28 12:10:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:09.408753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fef7e-dc03-4004-8d3b-690d69f07b2a None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:09.439640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fad91-45c9-4b32-a8d6-b2119c4621ee None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:09.461554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:09.477674 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5572f-da0e-42a9-88f6-215560f140bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:09.478282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5572f-da0e-42a9-88f6-215560f140bf None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:09.478560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5572f-da0e-42a9-88f6-215560f140bf None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:09.478833 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5572f-da0e-42a9-88f6-215560f140bf None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:09.479466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5572f-da0e-42a9-88f6-215560f140bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:09.479772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5572f-da0e-42a9-88f6-215560f140bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:09.481126 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c5572f-da0e-42a9-88f6-215560f140bf None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:09.493816 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 397/798] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:09.495022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20712da0-c774-48ca-a460-11add20c8c04 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:09.495310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20712da0-c774-48ca-a460-11add20c8c04 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:09.495582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20712da0-c774-48ca-a460-11add20c8c04 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:09.501099 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 398/799] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:09.501918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20712da0-c774-48ca-a460-11add20c8c04 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:09.521962 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fef7e-dc03-4004-8d3b-690d69f07b2a None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:09.556959 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e53fba-8d32-4f77-b5bf-b09e3ccf8913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:09.557604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e53fba-8d32-4f77-b5bf-b09e3ccf8913 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:09.557890 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e53fba-8d32-4f77-b5bf-b09e3ccf8913 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:09.558167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e53fba-8d32-4f77-b5bf-b09e3ccf8913 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:09.558812 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e53fba-8d32-4f77-b5bf-b09e3ccf8913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:09.559124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e53fba-8d32-4f77-b5bf-b09e3ccf8913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:09.560487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e53fba-8d32-4f77-b5bf-b09e3ccf8913 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:09.572472 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 402/800] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:09.573298 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e53fba-8d32-4f77-b5bf-b09e3ccf8913 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:09.592302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:09.616436 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20712da0-c774-48ca-a460-11add20c8c04 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:09.653038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310eb17-7363-4783-824a-be2cf9e1e485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:09.653668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310eb17-7363-4783-824a-be2cf9e1e485 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:09.653945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310eb17-7363-4783-824a-be2cf9e1e485 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:09.654214 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310eb17-7363-4783-824a-be2cf9e1e485 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:09.654821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310eb17-7363-4783-824a-be2cf9e1e485 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:09.655125 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310eb17-7363-4783-824a-be2cf9e1e485 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:09.657161 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310eb17-7363-4783-824a-be2cf9e1e485 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:09.661668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 399/801] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Sep 28 12:10:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:09.663930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edc5ed-0678-4930-b700-29b9acc63d07 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:09.664713 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edc5ed-0678-4930-b700-29b9acc63d07 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:09.665240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edc5ed-0678-4930-b700-29b9acc63d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:09.682448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e53fba-8d32-4f77-b5bf-b09e3ccf8913 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:09.684225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-a3edc5ed-0678-4930-b700-29b9acc63d07 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:09.685431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a3edc5ed-0678-4930-b700-29b9acc63d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:09.694911 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:09.720930 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef907f-49d8-49e0-8ef7-f8660f1e68f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:09.721622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef907f-49d8-49e0-8ef7-f8660f1e68f8 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:09.721896 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef907f-49d8-49e0-8ef7-f8660f1e68f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:09.722174 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef907f-49d8-49e0-8ef7-f8660f1e68f8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:09.729824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef907f-49d8-49e0-8ef7-f8660f1e68f8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:09.731265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef907f-49d8-49e0-8ef7-f8660f1e68f8 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:09.743836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-8aef907f-49d8-49e0-8ef7-f8660f1e68f8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:10:09.743836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:09.743836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:09.743836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:09.743836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:09.743836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:09.743836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:09.743836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:09.743836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:09.743836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:09.743836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:09.743836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:09.743836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:10:09.743836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:10:09.743836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:10:09.743836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:09.744863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:09.745868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:09.745868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:09.745868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:10:09.745868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:10:09.745868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:10:09.745868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:10:09.745868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:10:09.745868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:10:09.745868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:10:09.745868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:09.745868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 403/802] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 12:10:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:09.745868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef907f-49d8-49e0-8ef7-f8660f1e68f8 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:09.766076 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 400/803] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:09.775013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edc5ed-0678-4930-b700-29b9acc63d07 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:09.877669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef907f-49d8-49e0-8ef7-f8660f1e68f8 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:09.915438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf2277-99c2-4802-925a-d351c5cc7b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:09.916086 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf2277-99c2-4802-925a-d351c5cc7b55 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:09.916364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf2277-99c2-4802-925a-d351c5cc7b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:09.916637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf2277-99c2-4802-925a-d351c5cc7b55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:09.924666 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf2277-99c2-4802-925a-d351c5cc7b55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:09.926153 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf2277-99c2-4802-925a-d351c5cc7b55 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:09.941379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-f8bf2277-99c2-4802-925a-d351c5cc7b55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:09.941379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:09.941379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:09.941379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:09.941379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:09.941379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:09.941379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:09.941379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:09.941379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:09.941379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:09.941379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:09.941379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:09.941379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:10:09.941379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:10:09.941379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:10:09.942343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:10:09.942343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:09.942343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:09.942343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:09.942343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:09.942343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:09.942343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:09.942343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:09.942343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:09.942343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:09.942343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:09.942343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:09.942343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:09.942343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:09.942343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:09.942343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:09.942343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:09.943265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:10:09.943265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:10:09.943265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:10:09.943265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:10:09.943265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:10:09.943265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:10:09.943265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:09.943265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:09.943729 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 404/804] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:09.944607 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf2277-99c2-4802-925a-d351c5cc7b55 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:09.977331 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:09.983935 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:10.005545 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edc5ed-0678-4930-b700-29b9acc63d07 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:10.041236 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a0031-fc58-4951-bdfd-e3246f299833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:10.041785 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a0031-fc58-4951-bdfd-e3246f299833 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:10.042096 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a0031-fc58-4951-bdfd-e3246f299833 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:10.042368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a0031-fc58-4951-bdfd-e3246f299833 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:10.043008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a0031-fc58-4951-bdfd-e3246f299833 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:10.043295 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a0031-fc58-4951-bdfd-e3246f299833 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:10.044641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a0031-fc58-4951-bdfd-e3246f299833 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:10.050333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 401/805] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Sep 28 12:10:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:10.051368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d84862-3c59-4854-91c8-2b0a2c14a2db None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:10.051650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d84862-3c59-4854-91c8-2b0a2c14a2db None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:10.051919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d84862-3c59-4854-91c8-2b0a2c14a2db None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:10.053111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 402/806] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:10.053810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d84862-3c59-4854-91c8-2b0a2c14a2db None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:10.074006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf2277-99c2-4802-925a-d351c5cc7b55 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:10.111411 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92c19f-b554-4653-8f2f-221cb19b71e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:10.112009 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92c19f-b554-4653-8f2f-221cb19b71e4 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:10.112291 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92c19f-b554-4653-8f2f-221cb19b71e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:10.112555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92c19f-b554-4653-8f2f-221cb19b71e4 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:10.113312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92c19f-b554-4653-8f2f-221cb19b71e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:10.113613 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92c19f-b554-4653-8f2f-221cb19b71e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:10.115025 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92c19f-b554-4653-8f2f-221cb19b71e4 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:10.132682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 405/807] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 12:10:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:10.134786 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92c19f-b554-4653-8f2f-221cb19b71e4 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:10.157799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d84862-3c59-4854-91c8-2b0a2c14a2db None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:10.192447 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f3aca-0c2d-4957-9d04-79a44574875e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:10.193054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f3aca-0c2d-4957-9d04-79a44574875e None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:10.193368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f3aca-0c2d-4957-9d04-79a44574875e None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:10.193644 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f3aca-0c2d-4957-9d04-79a44574875e None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:10.194324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f3aca-0c2d-4957-9d04-79a44574875e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:10.194663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f3aca-0c2d-4957-9d04-79a44574875e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:10.196080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f3aca-0c2d-4957-9d04-79a44574875e None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:10.203029 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:10.208412 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 403/808] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:10:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:10.209206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f3aca-0c2d-4957-9d04-79a44574875e None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:10.230915 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:10.243775 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92c19f-b554-4653-8f2f-221cb19b71e4 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:10.277480 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4af9e-075f-4d00-bb5b-5ed31802b5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:10.278107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4af9e-075f-4d00-bb5b-5ed31802b5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:10.278391 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4af9e-075f-4d00-bb5b-5ed31802b5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:10.278669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4af9e-075f-4d00-bb5b-5ed31802b5d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:10.288617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4af9e-075f-4d00-bb5b-5ed31802b5d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:10:10.293537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-86c4af9e-075f-4d00-bb5b-5ed31802b5d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:10:10.293537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:10.293537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:10.293537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:10.293537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:10.293537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:10.293537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:10.293537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:10.293537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:10.293537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:10.293537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:10.293537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:10.293537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:10:10.293537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:10:10.293537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:10:10.293537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:10:10.294631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:10:10.294631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:10:10.294631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:10:10.294631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:10:10.294631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:10.294631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:10.294631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:10.294631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:10.294631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:10.294631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:10.294631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:10.294631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:10.294631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:10.294631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:10.294631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:10.294631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:10.295604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:10.295604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:10.295604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:10.295604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:10.295604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:10:10.295604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:10:10.295604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:10:10.295604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:10:10.295604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:10:10.295604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:10:10.295604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:10:10.295604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:10.295604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 406/809] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:10:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:10.296452 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e23261-32a7-4e6d-92fe-cc0da8781e63 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:10.296452 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e23261-32a7-4e6d-92fe-cc0da8781e63 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:10.296452 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e23261-32a7-4e6d-92fe-cc0da8781e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:10.307311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f3aca-0c2d-4957-9d04-79a44574875e None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:10.314074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-11e23261-32a7-4e6d-92fe-cc0da8781e63 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:10.315278 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-11e23261-32a7-4e6d-92fe-cc0da8781e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:10.340650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2694c8-78a3-4590-92df-7b68212c7caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:10.341315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2694c8-78a3-4590-92df-7b68212c7caf None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:10.341596 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2694c8-78a3-4590-92df-7b68212c7caf None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:10.341868 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2694c8-78a3-4590-92df-7b68212c7caf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:10.342519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2694c8-78a3-4590-92df-7b68212c7caf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:10.344577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2694c8-78a3-4590-92df-7b68212c7caf None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:10.365627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 404/810] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:10:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:10:10.366444 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2694c8-78a3-4590-92df-7b68212c7caf None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:10.401164 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 407/811] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:10.401164 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e23261-32a7-4e6d-92fe-cc0da8781e63 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:10.458940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:10.516688 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2694c8-78a3-4590-92df-7b68212c7caf None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:10.517622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:10.539460 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e23261-32a7-4e6d-92fe-cc0da8781e63 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:10.557066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b9424-4288-4896-9048-b3f65ad5286b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:10.557663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b9424-4288-4896-9048-b3f65ad5286b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:10.557941 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b9424-4288-4896-9048-b3f65ad5286b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:10.558205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b9424-4288-4896-9048-b3f65ad5286b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:10.579730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b46d78-e88e-41bc-928e-be4b076162e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:10.580631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b46d78-e88e-41bc-928e-be4b076162e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:10.581061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b46d78-e88e-41bc-928e-be4b076162e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:10.581572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b46d78-e88e-41bc-928e-be4b076162e2 None admin] PATH_INFO: `/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/975a8d7349994520bcbfb5f75e52aac7/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:10.583817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b46d78-e88e-41bc-928e-be4b076162e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6335d0db1f4844c39c255909189e0ce9, user_id=975a8d7349994520bcbfb5f75e52aac7, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:10.585545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b46d78-e88e-41bc-928e-be4b076162e2 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:10.603668 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 408/812] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:10:10 2022] PUT /identity/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/975a8d7349994520bcbfb5f75e52aac7/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:10:10.605661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b46d78-e88e-41bc-928e-be4b076162e2 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:10.641538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b9424-4288-4896-9048-b3f65ad5286b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:10.643178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b9424-4288-4896-9048-b3f65ad5286b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:10.709463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:10.760917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b46d78-e88e-41bc-928e-be4b076162e2 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:10.763398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 405/813] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:10:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:10:10.764395 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b9424-4288-4896-9048-b3f65ad5286b None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:10.816440 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17809a-4c4b-4807-a80c-47097213159e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:10.817102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17809a-4c4b-4807-a80c-47097213159e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:10.817546 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17809a-4c4b-4807-a80c-47097213159e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:10.818264 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17809a-4c4b-4807-a80c-47097213159e None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:10.818800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17809a-4c4b-4807-a80c-47097213159e None admin] RBAC: Adding query filter params (name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:10.819106 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17809a-4c4b-4807-a80c-47097213159e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:10.820516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17809a-4c4b-4807-a80c-47097213159e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:10.822060 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:10.836231 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 409/814] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:10:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:10.837254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17809a-4c4b-4807-a80c-47097213159e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:10.884370 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b9424-4288-4896-9048-b3f65ad5286b None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:10.920313 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8c103-5e5d-4cb4-94c8-3a47bd1a4634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:10.920313 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8c103-5e5d-4cb4-94c8-3a47bd1a4634 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:10.920665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8c103-5e5d-4cb4-94c8-3a47bd1a4634 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:10.920885 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8c103-5e5d-4cb4-94c8-3a47bd1a4634 None admin] PATH_INFO: `/v3/services/network` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:10.922045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8c103-5e5d-4cb4-94c8-3a47bd1a4634 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:10.923872 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8c103-5e5d-4cb4-94c8-3a47bd1a4634 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:10.933273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-4ad8c103-5e5d-4cb4-94c8-3a47bd1a4634 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 12:10:10.933273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:10.933273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:10.933273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:10.933273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:10.933273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:10.933273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:10.933273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:10.933273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 12:10:10.933273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:10.933273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:10.933273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:10.933273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:10.933273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:10.933273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:10.933273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:10.933273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:10.934298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:10.934298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:10.934298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:10.934298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:10.934298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:10.934298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:10.934298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:10.934298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:10.934298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:10.934298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:10.934298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:10.934298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:10.934298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:10.934298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:10.934298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:10.934298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:10.935133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:10.935133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:10.935133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:10.935133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:10.935133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:10.935133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:10.935133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:10.935133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:10.935133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:10.935133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:10.935133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:10.935133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:10.935133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 12:10:10.935133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:10.935133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 406/815] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:10 2022] GET /identity/v3/services/network => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:10.935934 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8c103-5e5d-4cb4-94c8-3a47bd1a4634 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:10.961907 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17809a-4c4b-4807-a80c-47097213159e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:10.981712 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:11.009473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431b84c-2a96-45b5-ad53-4b8ed6aac41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:11.011282 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431b84c-2a96-45b5-ad53-4b8ed6aac41d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:11.011713 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431b84c-2a96-45b5-ad53-4b8ed6aac41d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:11.012247 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431b84c-2a96-45b5-ad53-4b8ed6aac41d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:11.030433 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431b84c-2a96-45b5-ad53-4b8ed6aac41d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:11.033957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431b84c-2a96-45b5-ad53-4b8ed6aac41d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:11.053500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-8431b84c-2a96-45b5-ad53-4b8ed6aac41d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:11.053500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:11.053500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:11.053500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:11.053500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:11.053500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:11.053500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:11.053500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:11.053500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:11.053500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:11.053500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:11.053500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:11.053500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:11.053500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:11.053500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:11.053500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:11.054722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:11.054722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:11.054722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:11.054722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:11.054722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:11.054722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:11.054722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:11.054722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:11.054722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:11.054722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:11.054722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:11.054722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:11.054722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:11.054722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:11.054722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:11.054722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:11.054722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:11.055816 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:11.055816 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:11.055816 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:11.057797 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 410/816] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:11.058744 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8c103-5e5d-4cb4-94c8-3a47bd1a4634 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:11.059319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431b84c-2a96-45b5-ad53-4b8ed6aac41d None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:11.107509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a637090-32f2-4685-a24e-70b93cf022c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:11.108210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a637090-32f2-4685-a24e-70b93cf022c6 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:11.108583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a637090-32f2-4685-a24e-70b93cf022c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:11.109156 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a637090-32f2-4685-a24e-70b93cf022c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:11.121710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a637090-32f2-4685-a24e-70b93cf022c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:11.123271 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a637090-32f2-4685-a24e-70b93cf022c6 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:11.128820 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:11.143357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-0a637090-32f2-4685-a24e-70b93cf022c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:11.143357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:11.143357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:11.143357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:11.143357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:11.143357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:11.143357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:11.143357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:11.143357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:11.143357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:11.143357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:11.143357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:11.143357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:11.143357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:11.143357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:11.143357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:11.144543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:11.144543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:11.144543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:11.144543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:11.144543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:11.144543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:11.144543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:11.144543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:11.144543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:11.144543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:11.144543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:11.144543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:11.144543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:11.144543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:11.144543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:11.144543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:11.144543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:11.145641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:11.145641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:11.145641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:11.151976 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 407/817] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:11.151976 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a637090-32f2-4685-a24e-70b93cf022c6 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:11.210518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431b84c-2a96-45b5-ad53-4b8ed6aac41d None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:11.232160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:11.257621 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07f8d4-bd21-4a3b-b1e7-5a3029e1d865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:11.258740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07f8d4-bd21-4a3b-b1e7-5a3029e1d865 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:11.259161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07f8d4-bd21-4a3b-b1e7-5a3029e1d865 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:11.259577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07f8d4-bd21-4a3b-b1e7-5a3029e1d865 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:11.261002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07f8d4-bd21-4a3b-b1e7-5a3029e1d865 None admin] RBAC: Adding query filter params (name=network) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:11.261482 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07f8d4-bd21-4a3b-b1e7-5a3029e1d865 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:11.263521 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07f8d4-bd21-4a3b-b1e7-5a3029e1d865 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:11.270950 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 411/818] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 12:10:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:11.271753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a637090-32f2-4685-a24e-70b93cf022c6 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:11.272186 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07f8d4-bd21-4a3b-b1e7-5a3029e1d865 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:11.310460 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f8d84-c2da-4b8e-b082-59752b946350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:11.311343 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f8d84-c2da-4b8e-b082-59752b946350 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:11.311749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f8d84-c2da-4b8e-b082-59752b946350 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:11.312896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f8d84-c2da-4b8e-b082-59752b946350 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:11.313956 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f8d84-c2da-4b8e-b082-59752b946350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:11.314373 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f8d84-c2da-4b8e-b082-59752b946350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:11.315906 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f8d84-c2da-4b8e-b082-59752b946350 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:11.330953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 408/819] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:11.332201 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f8d84-c2da-4b8e-b082-59752b946350 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:11.409894 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07f8d4-bd21-4a3b-b1e7-5a3029e1d865 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:11.430083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:11.460958 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465ddf5-ac55-4933-8bc9-02fa8aade8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:11.462317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465ddf5-ac55-4933-8bc9-02fa8aade8cd None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:11.462873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465ddf5-ac55-4933-8bc9-02fa8aade8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:11.463290 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465ddf5-ac55-4933-8bc9-02fa8aade8cd None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:11.464142 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465ddf5-ac55-4933-8bc9-02fa8aade8cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:11.464586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465ddf5-ac55-4933-8bc9-02fa8aade8cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:11.464760 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f8d84-c2da-4b8e-b082-59752b946350 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:11.466516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465ddf5-ac55-4933-8bc9-02fa8aade8cd None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:11.488150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 412/820] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:11.488150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465ddf5-ac55-4933-8bc9-02fa8aade8cd None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:11.507386 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:11.516180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ce4fa-d729-40d5-9623-bf3e02eee848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:11.516986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ce4fa-d729-40d5-9623-bf3e02eee848 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:11.517808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ce4fa-d729-40d5-9623-bf3e02eee848 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:11.518583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ce4fa-d729-40d5-9623-bf3e02eee848 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:11.519400 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ce4fa-d729-40d5-9623-bf3e02eee848 None admin] RBAC: Adding query filter params (type=network) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:11.520059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ce4fa-d729-40d5-9623-bf3e02eee848 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:11.522399 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ce4fa-d729-40d5-9623-bf3e02eee848 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:11.529908 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 409/821] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 12:10:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:11.530953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ce4fa-d729-40d5-9623-bf3e02eee848 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:11.599590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465ddf5-ac55-4933-8bc9-02fa8aade8cd None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:11.636582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49224ab0-c37b-48ac-a560-ae5a0676f709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:11.637508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49224ab0-c37b-48ac-a560-ae5a0676f709 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:11.637781 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49224ab0-c37b-48ac-a560-ae5a0676f709 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:11.638158 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49224ab0-c37b-48ac-a560-ae5a0676f709 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:11.646052 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49224ab0-c37b-48ac-a560-ae5a0676f709 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:11.647714 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49224ab0-c37b-48ac-a560-ae5a0676f709 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:11.660552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-49224ab0-c37b-48ac-a560-ae5a0676f709 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:11.660552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:11.660552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:11.660552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:11.660552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:11.660552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:11.660552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:11.660552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:11.660552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:11.660552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:11.660552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:11.660552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:11.660552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:10:11.660552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:10:11.660552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:10:11.661652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:10:11.661652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:11.661652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:11.661652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:11.661652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:11.661652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:11.661652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:11.661652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:11.661652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:11.661652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:11.661652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:11.661652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:11.661652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:11.661652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:11.661652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:11.661652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:11.661652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:11.662727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:10:11.662727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:10:11.662727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:10:11.662727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:10:11.662727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:10:11.662727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:10:11.662727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:11.662727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:11.664360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 413/822] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:11.665285 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49224ab0-c37b-48ac-a560-ae5a0676f709 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:11.667379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ce4fa-d729-40d5-9623-bf3e02eee848 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:11.701318 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:11.715489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715b022-ef8d-4051-9c9f-e547c82563e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:11.716370 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715b022-ef8d-4051-9c9f-e547c82563e8 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:11.716829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715b022-ef8d-4051-9c9f-e547c82563e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:11.717267 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715b022-ef8d-4051-9c9f-e547c82563e8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:11.726234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e715b022-ef8d-4051-9c9f-e547c82563e8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:11.728479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e715b022-ef8d-4051-9c9f-e547c82563e8 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:11.744197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-e715b022-ef8d-4051-9c9f-e547c82563e8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 12:10:11.744197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:11.744197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:11.744197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:11.744197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:11.744197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:11.744197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:11.744197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:11.744197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:11.744197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:11.744197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:11.744197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:11.744197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:10:11.744197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:10:11.744197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:10:11.744197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:11.745248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:11.746318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:11.746318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:11.746318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:10:11.746318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:10:11.746318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:10:11.746318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:10:11.746318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:10:11.746318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:10:11.746318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 12:10:11.746318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:11.754247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 410/823] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 12:10:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:11.754247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715b022-ef8d-4051-9c9f-e547c82563e8 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:11.808957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49224ab0-c37b-48ac-a560-ae5a0676f709 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:11.817072 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:11.857265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3009bb2-7d5e-4260-a065-ee46d1fea4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:11.858735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3009bb2-7d5e-4260-a065-ee46d1fea4d0 None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:11.859940 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3009bb2-7d5e-4260-a065-ee46d1fea4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:11.860413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3009bb2-7d5e-4260-a065-ee46d1fea4d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:11.861847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3009bb2-7d5e-4260-a065-ee46d1fea4d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:11.864047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3009bb2-7d5e-4260-a065-ee46d1fea4d0 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:11.868098 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715b022-ef8d-4051-9c9f-e547c82563e8 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:11.893268 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 414/824] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:10:11 2022] POST /identity/v3/endpoints => generated 337 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:10:11.894625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3009bb2-7d5e-4260-a065-ee46d1fea4d0 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:11.918618 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27528026-9489-4b64-b2ed-c271c67dcc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:11.919176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27528026-9489-4b64-b2ed-c271c67dcc7a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:11.919515 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27528026-9489-4b64-b2ed-c271c67dcc7a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:11.919794 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27528026-9489-4b64-b2ed-c271c67dcc7a None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:11.920669 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27528026-9489-4b64-b2ed-c271c67dcc7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:11.920978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27528026-9489-4b64-b2ed-c271c67dcc7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:11.925196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27528026-9489-4b64-b2ed-c271c67dcc7a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:11.939270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 411/825] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:10:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:11.944860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27528026-9489-4b64-b2ed-c271c67dcc7a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:12.092746 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:12.111866 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3009bb2-7d5e-4260-a065-ee46d1fea4d0 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:12.141495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27528026-9489-4b64-b2ed-c271c67dcc7a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:12.156513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:12.169741 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7f3cc-edb0-4576-b03a-4b5c272d83eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:12.170293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7f3cc-edb0-4576-b03a-4b5c272d83eb None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:12.170575 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7f3cc-edb0-4576-b03a-4b5c272d83eb None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:12.170851 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7f3cc-edb0-4576-b03a-4b5c272d83eb None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:12.171764 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7f3cc-edb0-4576-b03a-4b5c272d83eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:12.171987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7f3cc-edb0-4576-b03a-4b5c272d83eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:12.174247 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7f3cc-edb0-4576-b03a-4b5c272d83eb None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:12.195412 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042be26-4fed-44eb-8acc-748b4542f22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:12.196404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042be26-4fed-44eb-8acc-748b4542f22f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:12.196966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042be26-4fed-44eb-8acc-748b4542f22f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:12.197465 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042be26-4fed-44eb-8acc-748b4542f22f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:12.198365 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042be26-4fed-44eb-8acc-748b4542f22f None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=982b82e8ac424318a934f9149fb91706) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:12.198880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042be26-4fed-44eb-8acc-748b4542f22f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:12.200464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042be26-4fed-44eb-8acc-748b4542f22f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:12.205855 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 415/826] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Sep 28 12:10:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:12.210277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 412/827] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:10:11 2022] GET /identity/v3/role_assignments?user.id=982b82e8ac424318a934f9149fb91706&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:12.211317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7f3cc-edb0-4576-b03a-4b5c272d83eb None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:12.324391 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7f3cc-edb0-4576-b03a-4b5c272d83eb None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:12.363121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69077ee-4844-43f3-a477-741a98ec6d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:12.363954 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69077ee-4844-43f3-a477-741a98ec6d30 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:12.364326 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69077ee-4844-43f3-a477-741a98ec6d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:12.364690 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69077ee-4844-43f3-a477-741a98ec6d30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:12.376518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69077ee-4844-43f3-a477-741a98ec6d30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:12.378207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69077ee-4844-43f3-a477-741a98ec6d30 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:12.386140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:12.400368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-f69077ee-4844-43f3-a477-741a98ec6d30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:12.400368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:12.400368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:12.400368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:12.400368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:12.400368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:12.400368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:12.400368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:12.400368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:12.400368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:12.400368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:12.400368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:12.400368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:12.400368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:12.400368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:12.400368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:12.401588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:12.401588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:12.401588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:12.401588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:12.401588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:12.401588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:12.401588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:12.401588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:12.401588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:12.401588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:12.401588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:12.401588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:12.401588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:12.401588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:12.401588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:12.401588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:12.401588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:12.402714 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:12.402714 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:12.402714 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:12.404078 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 416/828] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:12.410304 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042be26-4fed-44eb-8acc-748b4542f22f None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:12.521923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042be26-4fed-44eb-8acc-748b4542f22f None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:12.557120 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09e702-3059-4bf2-8df6-5591f1bd374b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:12.557896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09e702-3059-4bf2-8df6-5591f1bd374b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:12.558268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09e702-3059-4bf2-8df6-5591f1bd374b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:12.558634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09e702-3059-4bf2-8df6-5591f1bd374b None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:12.559383 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09e702-3059-4bf2-8df6-5591f1bd374b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:12.559781 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09e702-3059-4bf2-8df6-5591f1bd374b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:12.561248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09e702-3059-4bf2-8df6-5591f1bd374b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:12.573810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 413/829] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:12.579950 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69077ee-4844-43f3-a477-741a98ec6d30 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:12.684424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69077ee-4844-43f3-a477-741a98ec6d30 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:12.723857 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf516a-bf79-4fdc-8919-07424cb347e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:12.724516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf516a-bf79-4fdc-8919-07424cb347e5 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:12.724795 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf516a-bf79-4fdc-8919-07424cb347e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:12.725074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf516a-bf79-4fdc-8919-07424cb347e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:12.732633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf516a-bf79-4fdc-8919-07424cb347e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:12.734682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf516a-bf79-4fdc-8919-07424cb347e5 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:12.746651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-ebcf516a-bf79-4fdc-8919-07424cb347e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:12.746651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:12.746651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:12.746651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:12.746651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:12.746651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:12.746651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:12.746651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:12.746651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:12.746651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:12.746651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:12.746651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:12.746651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:10:12.746651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:10:12.746651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:10:12.747635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:10:12.747635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:12.747635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:12.747635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:12.747635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:12.747635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:12.747635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:12.747635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:12.747635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:12.747635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:12.747635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:12.747635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:12.747635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:12.747635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:12.747635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:12.747635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:12.747635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:12.749085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:10:12.749085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:10:12.749085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:10:12.749085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:10:12.749085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:10:12.749085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:10:12.749085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:12.749085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:12.749085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 417/830] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:12.753692 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09e702-3059-4bf2-8df6-5591f1bd374b None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:12.789507 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:12.818485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70d2fe-e00f-4238-be89-8ba2a0709e4f None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:12.818817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70d2fe-e00f-4238-be89-8ba2a0709e4f None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:12.819034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70d2fe-e00f-4238-be89-8ba2a0709e4f None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:12.820603 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 418/831] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:12.826106 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcec57c-e7df-483f-8a24-b042233aeff8 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:12.826464 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcec57c-e7df-483f-8a24-b042233aeff8 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:12.826825 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcec57c-e7df-483f-8a24-b042233aeff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:12.846754 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-2bcec57c-e7df-483f-8a24-b042233aeff8 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:12.848057 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2bcec57c-e7df-483f-8a24-b042233aeff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:12.862535 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09e702-3059-4bf2-8df6-5591f1bd374b None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:12.912527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed35c7-315b-4fb0-bcf7-9481648fb686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:12.913254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:12.913529 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed35c7-315b-4fb0-bcf7-9481648fb686 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:12.913811 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed35c7-315b-4fb0-bcf7-9481648fb686 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:12.914082 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed35c7-315b-4fb0-bcf7-9481648fb686 None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:12.914767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed35c7-315b-4fb0-bcf7-9481648fb686 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:12.915070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed35c7-315b-4fb0-bcf7-9481648fb686 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:12.916789 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed35c7-315b-4fb0-bcf7-9481648fb686 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:12.927161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 419/832] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:10:12.936686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 414/833] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:10:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:12.943244 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcec57c-e7df-483f-8a24-b042233aeff8 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:12.995217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185abc60-3d07-44e6-9e44-007aa19d22fc None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:12.995511 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185abc60-3d07-44e6-9e44-007aa19d22fc None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:12.995776 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185abc60-3d07-44e6-9e44-007aa19d22fc None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:12.999336 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 415/834] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:13.005006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e1e0b-b063-4526-bf1c-8f34460c317a None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:13.005317 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e1e0b-b063-4526-bf1c-8f34460c317a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:13.005592 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e1e0b-b063-4526-bf1c-8f34460c317a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:13.024592 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-3f6e1e0b-b063-4526-bf1c-8f34460c317a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:13.026003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-3f6e1e0b-b063-4526-bf1c-8f34460c317a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:13.056346 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcec57c-e7df-483f-8a24-b042233aeff8 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:13.093405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00b85b-8768-4ea9-a4b0-4f5b06672ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:13.094373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00b85b-8768-4ea9-a4b0-4f5b06672ff4 None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:13.094539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00b85b-8768-4ea9-a4b0-4f5b06672ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:13.094906 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f00b85b-8768-4ea9-a4b0-4f5b06672ff4 None admin] PATH_INFO: `/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/bc69cf267bea41bf890bc7551d84ad6f/roles/60963384f3e54d9cbfbc8d12023a6cd9` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:13.102390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 416/835] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:13.114078 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f00b85b-8768-4ea9-a4b0-4f5b06672ff4 None admin] RBAC: Authorizing `identity:create_grant(project_id=6335d0db1f4844c39c255909189e0ce9, user_id=bc69cf267bea41bf890bc7551d84ad6f, role_id=60963384f3e54d9cbfbc8d12023a6cd9)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:13.115677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f00b85b-8768-4ea9-a4b0-4f5b06672ff4 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:13.131334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 420/836] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:10:12 2022] PUT /identity/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/bc69cf267bea41bf890bc7551d84ad6f/roles/60963384f3e54d9cbfbc8d12023a6cd9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:10:13.226075 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e1e0b-b063-4526-bf1c-8f34460c317a None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:13.255170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:13.339237 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e1e0b-b063-4526-bf1c-8f34460c317a None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:13.376631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5595ff-a2bf-44d1-8dc8-4b8ec2e10914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:13.377649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5595ff-a2bf-44d1-8dc8-4b8ec2e10914 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:13.378137 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5595ff-a2bf-44d1-8dc8-4b8ec2e10914 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:13.378612 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5595ff-a2bf-44d1-8dc8-4b8ec2e10914 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:13.379414 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5595ff-a2bf-44d1-8dc8-4b8ec2e10914 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:13.380974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5595ff-a2bf-44d1-8dc8-4b8ec2e10914 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:13.388849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00b85b-8768-4ea9-a4b0-4f5b06672ff4 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:13.391422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-7d5595ff-a2bf-44d1-8dc8-4b8ec2e10914 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 12:10:13.391422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:13.391422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:13.391422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:13.391422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:13.391422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:13.391422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:13.391422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:13.391422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 12:10:13.391422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:13.391422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:13.391422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:13.391422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:13.391422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:13.391422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:13.391422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:13.391422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:13.392411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:13.392411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:13.392411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:13.392411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:13.392411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:13.392411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:13.392411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:13.392411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:13.392411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:13.392411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:13.392411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:13.392411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:13.392411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:13.392411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:13.392411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:13.392411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:13.393288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:13.393288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:13.393288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:13.393288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:13.393288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:13.393288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:13.393288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:13.393288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:13.393288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:13.393288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:13.393288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:13.393288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:13.393288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 12:10:13.393288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:13.395523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 417/837] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:10:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:13.401315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5595ff-a2bf-44d1-8dc8-4b8ec2e10914 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:13.432856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:13.504470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:13.506271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f00b85b-8768-4ea9-a4b0-4f5b06672ff4 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:13.517176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5595ff-a2bf-44d1-8dc8-4b8ec2e10914 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:13.543340 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2db75-be09-477f-a60f-03d5671bc06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:13.544025 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2db75-be09-477f-a60f-03d5671bc06b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:13.544307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2db75-be09-477f-a60f-03d5671bc06b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:13.544578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2db75-be09-477f-a60f-03d5671bc06b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:13.555123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575586e-0dc6-43a5-830a-c2ebd3344be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:13.555287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2db75-be09-477f-a60f-03d5671bc06b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:10:13.555843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575586e-0dc6-43a5-830a-c2ebd3344be4 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:13.556184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575586e-0dc6-43a5-830a-c2ebd3344be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:13.556525 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575586e-0dc6-43a5-830a-c2ebd3344be4 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:13.557243 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575586e-0dc6-43a5-830a-c2ebd3344be4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:13.557614 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575586e-0dc6-43a5-830a-c2ebd3344be4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:13.559044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575586e-0dc6-43a5-830a-c2ebd3344be4 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:13.560188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-37e2db75-be09-477f-a60f-03d5671bc06b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:10:13.560188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:13.560188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:13.560188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:13.560188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:13.560188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:13.560188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:13.560188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:13.560188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:13.560188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:13.560188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:13.560188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:13.560188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:10:13.560188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:10:13.560188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:10:13.560188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:10:13.561124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:10:13.561124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:10:13.561124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:10:13.561124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:10:13.561124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:13.561124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:13.561124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:13.561124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:13.561124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:13.561124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:13.561124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:13.561124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:13.561124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:13.561124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:13.561124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:13.561124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:13.562550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:13.562550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:13.562550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:13.562550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:13.562550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:10:13.562550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:10:13.562550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:10:13.562550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:10:13.562550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:10:13.562550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:10:13.562550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:10:13.562550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:13.566972 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 421/838] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:10:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:13.569428 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 418/839] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:10:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:13.570295 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575586e-0dc6-43a5-830a-c2ebd3344be4 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:13.571530 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2db75-be09-477f-a60f-03d5671bc06b None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:13.673039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:13.677377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2db75-be09-477f-a60f-03d5671bc06b None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:13.679643 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575586e-0dc6-43a5-830a-c2ebd3344be4 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:13.714992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e838b33-7906-47b4-a29c-b3ac333e475c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:13.715763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e838b33-7906-47b4-a29c-b3ac333e475c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:13.716164 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e838b33-7906-47b4-a29c-b3ac333e475c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:13.716536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e838b33-7906-47b4-a29c-b3ac333e475c None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:13.717218 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6813621-0fd3-4552-a5d7-7b907f261200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:13.717372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e838b33-7906-47b4-a29c-b3ac333e475c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:13.717778 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e838b33-7906-47b4-a29c-b3ac333e475c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:13.718357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6813621-0fd3-4552-a5d7-7b907f261200 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:13.718780 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6813621-0fd3-4552-a5d7-7b907f261200 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:13.719279 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6813621-0fd3-4552-a5d7-7b907f261200 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:13.719893 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e838b33-7906-47b4-a29c-b3ac333e475c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:13.720348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6813621-0fd3-4552-a5d7-7b907f261200 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:13.720802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6813621-0fd3-4552-a5d7-7b907f261200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:13.722617 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6813621-0fd3-4552-a5d7-7b907f261200 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:13.729165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 422/840] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:10:13 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:13.736767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 419/841] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:10:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:13.742564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e838b33-7906-47b4-a29c-b3ac333e475c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:13.857886 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e838b33-7906-47b4-a29c-b3ac333e475c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:13.897268 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3359d-30ce-4c0e-8a34-ca525c7d8d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:13.897919 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3359d-30ce-4c0e-8a34-ca525c7d8d03 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:13.898255 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3359d-30ce-4c0e-8a34-ca525c7d8d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:13.898533 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3359d-30ce-4c0e-8a34-ca525c7d8d03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:13.910083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3359d-30ce-4c0e-8a34-ca525c7d8d03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:13.911537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3359d-30ce-4c0e-8a34-ca525c7d8d03 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:13.927614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-1ed3359d-30ce-4c0e-8a34-ca525c7d8d03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:13.927614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:13.927614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:13.927614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:13.927614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:13.927614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:13.927614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:13.927614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:13.927614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:13.927614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:13.927614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:13.927614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:13.927614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:13.927614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:13.927614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:13.927614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:13.928631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:13.928631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:13.928631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:13.928631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:13.928631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:13.928631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:13.928631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:13.928631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:13.928631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:13.928631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:13.928631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:13.928631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:13.928631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:13.928631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:13.928631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:13.928631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:13.928631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:13.929556 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:13.929556 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:13.929556 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:13.929556 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 423/842] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:13.934733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6813621-0fd3-4552-a5d7-7b907f261200 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:13.951102 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:14.047027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6813621-0fd3-4552-a5d7-7b907f261200 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:14.082355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43efd7-4cd0-4ae6-8eea-693024a9f456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:14.082970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43efd7-4cd0-4ae6-8eea-693024a9f456 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:14.083304 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43efd7-4cd0-4ae6-8eea-693024a9f456 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:14.083582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a43efd7-4cd0-4ae6-8eea-693024a9f456 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:14.084233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43efd7-4cd0-4ae6-8eea-693024a9f456 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:14.084603 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43efd7-4cd0-4ae6-8eea-693024a9f456 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:14.086064 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a43efd7-4cd0-4ae6-8eea-693024a9f456 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:14.100999 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 420/843] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:14.106356 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3359d-30ce-4c0e-8a34-ca525c7d8d03 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:14.123698 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:14.216936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3359d-30ce-4c0e-8a34-ca525c7d8d03 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:14.253858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747f6cd-73a9-4c71-84a7-ca5f106244a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:14.254494 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747f6cd-73a9-4c71-84a7-ca5f106244a1 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:14.254773 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747f6cd-73a9-4c71-84a7-ca5f106244a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:14.255046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747f6cd-73a9-4c71-84a7-ca5f106244a1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:14.263025 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747f6cd-73a9-4c71-84a7-ca5f106244a1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:14.264473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b747f6cd-73a9-4c71-84a7-ca5f106244a1 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:14.277951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-b747f6cd-73a9-4c71-84a7-ca5f106244a1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 12:10:14.277951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:14.277951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:14.277951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:14.277951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:14.277951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:14.277951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:14.277951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:14.277951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:14.277951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:14.277951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:14.277951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:14.277951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:10:14.277951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:10:14.277951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:10:14.277951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:14.278904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:14.279849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:14.279849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:14.279849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:10:14.279849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:10:14.279849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:10:14.279849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:10:14.279849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:10:14.279849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:10:14.279849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 12:10:14.279849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:14.279849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 424/844] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:10:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:14.285601 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43efd7-4cd0-4ae6-8eea-693024a9f456 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:14.365453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:14.391443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a43efd7-4cd0-4ae6-8eea-693024a9f456 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:14.425908 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68355da-17e6-45d0-9510-8743e0df1f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:14.426514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68355da-17e6-45d0-9510-8743e0df1f1d None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:14.426788 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68355da-17e6-45d0-9510-8743e0df1f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:14.427053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68355da-17e6-45d0-9510-8743e0df1f1d None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:14.427820 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68355da-17e6-45d0-9510-8743e0df1f1d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:14.428121 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68355da-17e6-45d0-9510-8743e0df1f1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:14.429615 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68355da-17e6-45d0-9510-8743e0df1f1d None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:14.449314 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 421/845] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Sep 28 12:10:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:14.454939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747f6cd-73a9-4c71-84a7-ca5f106244a1 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:14.548797 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:14.577105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747f6cd-73a9-4c71-84a7-ca5f106244a1 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:14.592185 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d982ab-8787-4e8b-819b-f82fc6263f19 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:14.592472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d982ab-8787-4e8b-819b-f82fc6263f19 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:14.592733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d982ab-8787-4e8b-819b-f82fc6263f19 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:14.595204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 422/846] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:14.605247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a2815-ae0b-405a-aff9-74a8fcf23475 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:14.605795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a2815-ae0b-405a-aff9-74a8fcf23475 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:14.606264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a2815-ae0b-405a-aff9-74a8fcf23475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:14.620730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd9644-0d6b-45a2-ab92-bbc49c941e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:14.621379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd9644-0d6b-45a2-ab92-bbc49c941e7a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:14.621661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd9644-0d6b-45a2-ab92-bbc49c941e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:14.621936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd9644-0d6b-45a2-ab92-bbc49c941e7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:14.625748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-d13a2815-ae0b-405a-aff9-74a8fcf23475 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:14.627877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d13a2815-ae0b-405a-aff9-74a8fcf23475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:14.694505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd9644-0d6b-45a2-ab92-bbc49c941e7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:14.696298 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd9644-0d6b-45a2-ab92-bbc49c941e7a None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:14.709307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 423/847] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:10:14.784314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:14.811995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 425/848] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:10:14 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:10:14.818288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a2815-ae0b-405a-aff9-74a8fcf23475 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:14.888038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:14.927372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a2815-ae0b-405a-aff9-74a8fcf23475 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:14.961401 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe7f86-4158-482f-8895-f78425d92563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:14.962035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe7f86-4158-482f-8895-f78425d92563 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:14.962312 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe7f86-4158-482f-8895-f78425d92563 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:14.962580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe7f86-4158-482f-8895-f78425d92563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:14.973515 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe7f86-4158-482f-8895-f78425d92563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:14.974954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe7f86-4158-482f-8895-f78425d92563 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:14.990101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-8fbe7f86-4158-482f-8895-f78425d92563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:14.990101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:14.990101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:14.990101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:14.990101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:14.990101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:14.990101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:14.990101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:14.990101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:14.990101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:14.990101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:14.990101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:14.990101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:14.990101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:14.990101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:14.990101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:14.991044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:14.991044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:14.991044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:14.991044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:14.991044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:14.991044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:14.991044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:14.991044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:14.991044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:14.991044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:14.991044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:14.991044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:14.991044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:14.991044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:14.991044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:14.991044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:14.991044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:14.992308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:14.992308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:14.992308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:14.992308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 424/849] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:14.996865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd9644-0d6b-45a2-ab92-bbc49c941e7a None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:15.005006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe7f86-4158-482f-8895-f78425d92563 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:15.108658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd9644-0d6b-45a2-ab92-bbc49c941e7a None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:15.131456 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe7f86-4158-482f-8895-f78425d92563 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:15.147294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e7bd0-3e85-47e0-a1f8-295f84d688fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:15.148237 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e7bd0-3e85-47e0-a1f8-295f84d688fa None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:15.148738 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e7bd0-3e85-47e0-a1f8-295f84d688fa None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:15.149328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e7bd0-3e85-47e0-a1f8-295f84d688fa None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:15.150274 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e7bd0-3e85-47e0-a1f8-295f84d688fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:15.150753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e7bd0-3e85-47e0-a1f8-295f84d688fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:15.151333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:15.153689 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e7bd0-3e85-47e0-a1f8-295f84d688fa None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:15.171976 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 426/850] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:15.177432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e7bd0-3e85-47e0-a1f8-295f84d688fa None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:15.184394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775a62a-e5e7-41ee-9c7d-87c17b18200a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:15.184394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775a62a-e5e7-41ee-9c7d-87c17b18200a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:15.184394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775a62a-e5e7-41ee-9c7d-87c17b18200a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:15.184394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c775a62a-e5e7-41ee-9c7d-87c17b18200a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:15.184742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c775a62a-e5e7-41ee-9c7d-87c17b18200a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:15.185946 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c775a62a-e5e7-41ee-9c7d-87c17b18200a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:15.198486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-c775a62a-e5e7-41ee-9c7d-87c17b18200a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 12:10:15.198486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:15.198486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:15.198486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:15.198486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:15.198486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:15.198486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:15.198486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:15.198486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 12:10:15.198486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:15.198486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:15.198486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:15.198486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:15.198486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:15.198486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:15.198486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:15.199443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:15.199443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:15.199443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:15.199443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:15.199443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:15.199443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:15.199443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:15.199443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:15.199443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:15.199443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:15.199443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:15.199443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:15.199443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:15.199443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:15.199443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:15.199443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:15.199443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:15.205650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:15.205650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:15.205650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:15.205650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:15.205650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:15.205650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:15.205650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:15.205650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:15.205650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:15.205650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:15.205650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:15.205650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:15.205650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 12:10:15.205650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:15.205650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 425/851] 10.222.0.60 () {56 vars in 1132 bytes} [Wed Sep 28 12:10:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:15.208439 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775a62a-e5e7-41ee-9c7d-87c17b18200a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:15.225256 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:15.301267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e7bd0-3e85-47e0-a1f8-295f84d688fa None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:15.346937 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35714196-9613-48d3-bcd3-1db61ca3d49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:15.347625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35714196-9613-48d3-bcd3-1db61ca3d49d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:15.348214 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35714196-9613-48d3-bcd3-1db61ca3d49d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:15.348495 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35714196-9613-48d3-bcd3-1db61ca3d49d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:15.351105 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c775a62a-e5e7-41ee-9c7d-87c17b18200a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:15.356937 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35714196-9613-48d3-bcd3-1db61ca3d49d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:15.358413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35714196-9613-48d3-bcd3-1db61ca3d49d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:15.389006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988522e-2d61-4770-87af-0bbb31865736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:15.389998 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988522e-2d61-4770-87af-0bbb31865736 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:15.391006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988522e-2d61-4770-87af-0bbb31865736 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:15.391508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988522e-2d61-4770-87af-0bbb31865736 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:15.392366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988522e-2d61-4770-87af-0bbb31865736 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:15.392899 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988522e-2d61-4770-87af-0bbb31865736 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:15.394540 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988522e-2d61-4770-87af-0bbb31865736 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:15.404965 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 426/852] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 12:10:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:15.406821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988522e-2d61-4770-87af-0bbb31865736 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:15.455712 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:15.495122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-35714196-9613-48d3-bcd3-1db61ca3d49d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:15.495122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:15.495122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:15.495122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:15.495122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:15.495122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:15.495122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:15.495122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:15.495122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:15.495122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:15.495122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:15.495122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:15.495122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:10:15.495122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:10:15.495122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:10:15.496259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:10:15.496259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:15.496259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:15.496259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:15.496259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:15.496259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:15.496259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:15.496259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:15.496259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:15.496259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:15.496259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:15.496259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:15.496259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:15.496259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:15.496259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:15.496259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:15.496259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:15.497179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:10:15.497179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:10:15.497179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:10:15.497179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:10:15.497179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:10:15.497179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:10:15.497179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:15.497179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:15.497179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 427/853] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:15.497750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead80e81-add7-424a-a6a1-074453fb5ec2 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:15.497750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead80e81-add7-424a-a6a1-074453fb5ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:15.497908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead80e81-add7-424a-a6a1-074453fb5ec2 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:15.501201 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 428/854] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:15.503138 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead80e81-add7-424a-a6a1-074453fb5ec2 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:15.525823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988522e-2d61-4770-87af-0bbb31865736 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:15.561535 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc85f1-53ed-4db6-994d-bdacae00f549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:15.562695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc85f1-53ed-4db6-994d-bdacae00f549 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:15.562975 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc85f1-53ed-4db6-994d-bdacae00f549 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:15.563247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc85f1-53ed-4db6-994d-bdacae00f549 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:15.563876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc85f1-53ed-4db6-994d-bdacae00f549 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:15.564176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc85f1-53ed-4db6-994d-bdacae00f549 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:15.565595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc85f1-53ed-4db6-994d-bdacae00f549 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:15.578839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 427/855] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 12:10:15 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:15.578839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c021a18-da43-4713-b262-6dc2fa87c072 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:15.578839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c021a18-da43-4713-b262-6dc2fa87c072 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:15.578839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c021a18-da43-4713-b262-6dc2fa87c072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:15.595000 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-0c021a18-da43-4713-b262-6dc2fa87c072 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:15.596630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-0c021a18-da43-4713-b262-6dc2fa87c072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:15.625960 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:15.637630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead80e81-add7-424a-a6a1-074453fb5ec2 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:15.690588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb98508-f696-4705-80eb-fb603ea3ec1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:15.691798 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb98508-f696-4705-80eb-fb603ea3ec1a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:15.692275 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb98508-f696-4705-80eb-fb603ea3ec1a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:15.692819 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb98508-f696-4705-80eb-fb603ea3ec1a None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:15.693893 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb98508-f696-4705-80eb-fb603ea3ec1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:15.694427 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb98508-f696-4705-80eb-fb603ea3ec1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:15.701159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 428/856] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:15.701341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb98508-f696-4705-80eb-fb603ea3ec1a None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:15.712541 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 429/857] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:10:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:15.719636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c021a18-da43-4713-b262-6dc2fa87c072 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:15.763830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:15.834519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c021a18-da43-4713-b262-6dc2fa87c072 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:15.871402 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ba685-b25d-4c20-9640-e645afbc627d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:15.872011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ba685-b25d-4c20-9640-e645afbc627d None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:15.872287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ba685-b25d-4c20-9640-e645afbc627d None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:15.872556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ba685-b25d-4c20-9640-e645afbc627d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:15.873303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ba685-b25d-4c20-9640-e645afbc627d None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=975a8d7349994520bcbfb5f75e52aac7) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:15.873612 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ba685-b25d-4c20-9640-e645afbc627d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:15.875112 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ba685-b25d-4c20-9640-e645afbc627d None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:15.881453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 429/858] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:10:15 2022] GET /identity/v3/role_assignments?user.id=975a8d7349994520bcbfb5f75e52aac7&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:15.989238 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb98508-f696-4705-80eb-fb603ea3ec1a None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:16.052347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5a95c-87a4-45d3-a600-f6081c221980 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:16.052347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5a95c-87a4-45d3-a600-f6081c221980 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:16.052655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5a95c-87a4-45d3-a600-f6081c221980 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:16.054087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 430/859] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:10:16.059412 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd87379-e12a-463b-aeb4-be1fe5d8197d None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:16.059701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd87379-e12a-463b-aeb4-be1fe5d8197d None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:16.059973 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd87379-e12a-463b-aeb4-be1fe5d8197d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:16.081323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-4bd87379-e12a-463b-aeb4-be1fe5d8197d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:16.082514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-4bd87379-e12a-463b-aeb4-be1fe5d8197d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:16.106971 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb98508-f696-4705-80eb-fb603ea3ec1a None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:16.145590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c4cb5-2475-43c1-a7cc-0cbcd178b2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:16.146226 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c4cb5-2475-43c1-a7cc-0cbcd178b2e0 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:16.146488 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c4cb5-2475-43c1-a7cc-0cbcd178b2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:16.146815 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c4cb5-2475-43c1-a7cc-0cbcd178b2e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:16.158734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c4cb5-2475-43c1-a7cc-0cbcd178b2e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:10:16.163743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 431/860] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:10:16.165876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-3f0c4cb5-2475-43c1-a7cc-0cbcd178b2e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:10:16.165876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:16.165876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:16.165876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:16.165876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:16.165876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:16.165876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:16.165876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:16.165876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:16.165876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:16.165876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:16.165876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:16.165876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:10:16.165876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:10:16.165876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:10:16.165876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:10:16.166800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:10:16.166800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:10:16.166800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:10:16.166800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:10:16.166800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:16.166800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:16.166800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:16.166800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:16.166800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:16.166800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:16.166800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:16.166800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:16.166800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:16.166800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:16.166800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:16.166800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:16.167667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:16.167667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:16.167667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:16.167667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:16.167667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:10:16.167667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:10:16.167667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:10:16.167667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:10:16.167667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:10:16.167667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:10:16.167667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:10:16.167667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:16.172882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 430/861] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:10:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:16.175963 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd87379-e12a-463b-aeb4-be1fe5d8197d None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:16.210593 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:16.283919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd87379-e12a-463b-aeb4-be1fe5d8197d None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:16.318962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6b919-8960-4443-a4ed-9ea76f9dc9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:16.319577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6b919-8960-4443-a4ed-9ea76f9dc9bf None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:16.319858 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6b919-8960-4443-a4ed-9ea76f9dc9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:16.320136 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6b919-8960-4443-a4ed-9ea76f9dc9bf None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:16.320799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6b919-8960-4443-a4ed-9ea76f9dc9bf None admin] RBAC: Adding query filter params (name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:16.321102 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6b919-8960-4443-a4ed-9ea76f9dc9bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:16.322492 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6b919-8960-4443-a4ed-9ea76f9dc9bf None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:16.331904 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 432/862] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:10:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:16.337834 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c4cb5-2475-43c1-a7cc-0cbcd178b2e0 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:16.355149 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:16.448498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c4cb5-2475-43c1-a7cc-0cbcd178b2e0 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:16.478381 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6b919-8960-4443-a4ed-9ea76f9dc9bf None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:16.485600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693114f-b4e9-4841-bedc-555b18a2088a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:16.486494 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693114f-b4e9-4841-bedc-555b18a2088a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:16.486774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693114f-b4e9-4841-bedc-555b18a2088a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:16.487047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9693114f-b4e9-4841-bedc-555b18a2088a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:16.501289 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9693114f-b4e9-4841-bedc-555b18a2088a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:16.502757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9693114f-b4e9-4841-bedc-555b18a2088a None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:16.524053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-9693114f-b4e9-4841-bedc-555b18a2088a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:16.524053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:16.524053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:16.524053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:16.524053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:16.524053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:16.524053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:16.524053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:16.524053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:16.524053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:16.524053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:16.524053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:16.524053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:16.524053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:16.524053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:16.524053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:16.525096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:16.525096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:16.525096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:16.525096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:16.525096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:16.525096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:16.525096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:16.525096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:16.525096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:16.525096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:16.525096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:16.525096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:16.525096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:16.525096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:16.525096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:16.525096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:16.525096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:16.526025 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:16.526025 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:16.526025 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:16.528510 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 431/863] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:16.534991 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693114f-b4e9-4841-bedc-555b18a2088a None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:16.596979 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:16.606886 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6b919-8960-4443-a4ed-9ea76f9dc9bf None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:16.613443 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:16.644722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3245d3-af78-4543-9f3a-3ab5a75ac13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:16.645355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3245d3-af78-4543-9f3a-3ab5a75ac13f None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:16.645637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3245d3-af78-4543-9f3a-3ab5a75ac13f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:16.646051 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3245d3-af78-4543-9f3a-3ab5a75ac13f None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:16.646700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3245d3-af78-4543-9f3a-3ab5a75ac13f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:16.648657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3245d3-af78-4543-9f3a-3ab5a75ac13f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:16.650493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9693114f-b4e9-4841-bedc-555b18a2088a None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:16.665594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 433/864] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 12:10:16 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:10:16.696047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda7882-4b80-4a8d-8b95-a47a4b48c7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:16.696998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda7882-4b80-4a8d-8b95-a47a4b48c7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:16.697524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda7882-4b80-4a8d-8b95-a47a4b48c7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:16.697975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda7882-4b80-4a8d-8b95-a47a4b48c7a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:16.698838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda7882-4b80-4a8d-8b95-a47a4b48c7a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:16.699373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda7882-4b80-4a8d-8b95-a47a4b48c7a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:16.701046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda7882-4b80-4a8d-8b95-a47a4b48c7a3 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:16.722908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 432/865] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:16.728732 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3245d3-af78-4543-9f3a-3ab5a75ac13f None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:16.845667 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3245d3-af78-4543-9f3a-3ab5a75ac13f None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:16.880324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7602e-2949-46b1-bd51-7147ec9c90d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:16.880962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7602e-2949-46b1-bd51-7147ec9c90d9 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:16.881272 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7602e-2949-46b1-bd51-7147ec9c90d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:16.881546 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7602e-2949-46b1-bd51-7147ec9c90d9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:16.890511 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7602e-2949-46b1-bd51-7147ec9c90d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:16.892470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7602e-2949-46b1-bd51-7147ec9c90d9 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:16.904038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-49e7602e-2949-46b1-bd51-7147ec9c90d9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 12:10:16.904038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:16.904038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:16.904038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:16.904038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:16.904038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:16.904038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:16.904038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:16.904038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:16.904038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:16.904038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:16.904038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:16.904038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:10:16.904038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:10:16.904038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:10:16.904038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:16.904997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:16.906528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:16.906528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:16.906528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:10:16.906528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:10:16.906528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:10:16.906528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:10:16.906528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:10:16.906528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:10:16.906528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 12:10:16.906528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:16.909884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 434/866] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 12:10:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:16.911944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda7882-4b80-4a8d-8b95-a47a4b48c7a3 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:17.018585 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda7882-4b80-4a8d-8b95-a47a4b48c7a3 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:17.043183 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:17.056181 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f536c-8240-47af-bb1c-414ae104c956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:17.057232 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f536c-8240-47af-bb1c-414ae104c956 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:17.057708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f536c-8240-47af-bb1c-414ae104c956 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:17.058171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f536c-8240-47af-bb1c-414ae104c956 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:17.059103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f536c-8240-47af-bb1c-414ae104c956 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:17.059616 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f536c-8240-47af-bb1c-414ae104c956 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:17.061250 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f536c-8240-47af-bb1c-414ae104c956 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:17.080049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 433/867] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Sep 28 12:10:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:17.086254 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7602e-2949-46b1-bd51-7147ec9c90d9 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:17.102704 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:17.196013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7602e-2949-46b1-bd51-7147ec9c90d9 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:17.231742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899de68-a667-4a41-bedd-3e2110d132ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:17.232348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899de68-a667-4a41-bedd-3e2110d132ad None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:17.232625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899de68-a667-4a41-bedd-3e2110d132ad None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:17.232895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899de68-a667-4a41-bedd-3e2110d132ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:17.305763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899de68-a667-4a41-bedd-3e2110d132ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:17.307248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899de68-a667-4a41-bedd-3e2110d132ad None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:17.322003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:17.416346 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 435/868] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:10:17 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:10:17.422757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f536c-8240-47af-bb1c-414ae104c956 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:17.528328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f536c-8240-47af-bb1c-414ae104c956 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:17.564091 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf838b-51f9-4e2c-a75e-4f8327a57f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:17.565088 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf838b-51f9-4e2c-a75e-4f8327a57f0e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:17.565588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf838b-51f9-4e2c-a75e-4f8327a57f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:17.566055 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf838b-51f9-4e2c-a75e-4f8327a57f0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:17.576446 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf838b-51f9-4e2c-a75e-4f8327a57f0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:17.578764 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf838b-51f9-4e2c-a75e-4f8327a57f0e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:17.594206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-0acf838b-51f9-4e2c-a75e-4f8327a57f0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:17.594206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:17.594206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:17.594206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:17.594206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:17.594206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:17.594206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:17.594206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:17.594206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:17.594206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:17.594206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:17.594206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:17.594206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:17.594206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:17.594206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:17.594206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:17.595140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:17.595140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:17.595140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:17.595140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:17.595140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:17.595140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:17.595140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:17.595140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:17.595140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:17.595140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:17.595140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:17.595140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:17.595140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:17.595140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:17.595140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:17.595140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:17.595140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:17.596015 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:17.596015 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:17.596015 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:17.597768 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 434/869] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:17.603445 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899de68-a667-4a41-bedd-3e2110d132ad None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:17.711840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899de68-a667-4a41-bedd-3e2110d132ad None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:17.729182 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:17.754170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a69769-04ce-4642-b48b-671673addea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:17.754802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69769-04ce-4642-b48b-671673addea9 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:17.755080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69769-04ce-4642-b48b-671673addea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:17.755352 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a69769-04ce-4642-b48b-671673addea9 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:17.756007 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a69769-04ce-4642-b48b-671673addea9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:17.756309 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a69769-04ce-4642-b48b-671673addea9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:17.757757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a69769-04ce-4642-b48b-671673addea9 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:17.771060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 436/870] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:17.776515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf838b-51f9-4e2c-a75e-4f8327a57f0e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:17.810506 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:17.854354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308de16a-f6ae-43aa-a1e2-f012bed3ab59 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:17.854683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308de16a-f6ae-43aa-a1e2-f012bed3ab59 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:17.854983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308de16a-f6ae-43aa-a1e2-f012bed3ab59 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:17.856427 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 437/871] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:10:17.862163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3975bd-42da-4afd-aaa8-24443aa923dd None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:17.862450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3975bd-42da-4afd-aaa8-24443aa923dd None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:17.862725 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3975bd-42da-4afd-aaa8-24443aa923dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:17.882078 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf838b-51f9-4e2c-a75e-4f8327a57f0e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:17.885402 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-fe3975bd-42da-4afd-aaa8-24443aa923dd None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:17.886582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-fe3975bd-42da-4afd-aaa8-24443aa923dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:17.917225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766db85-c2d6-4bd3-92b5-1e165005f48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:17.917888 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766db85-c2d6-4bd3-92b5-1e165005f48c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:17.918176 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766db85-c2d6-4bd3-92b5-1e165005f48c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:17.918551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766db85-c2d6-4bd3-92b5-1e165005f48c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:17.925934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766db85-c2d6-4bd3-92b5-1e165005f48c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:17.927407 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766db85-c2d6-4bd3-92b5-1e165005f48c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:17.940054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-6766db85-c2d6-4bd3-92b5-1e165005f48c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:17.940054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:17.940054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:17.940054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:17.940054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:17.940054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:17.940054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:17.940054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:17.940054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:17.940054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:17.940054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:17.940054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:17.940054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:10:17.940054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:10:17.940054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:10:17.940965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:10:17.940965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:17.940965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:17.940965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:17.940965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:17.940965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:17.940965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:17.940965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:17.940965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:17.940965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:17.940965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:17.940965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:17.940965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:17.940965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:17.940965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:17.940965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:17.940965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:17.941877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:10:17.941877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:10:17.941877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:10:17.941877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:10:17.941877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:10:17.941877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:10:17.941877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:17.941877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:17.941877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 435/872] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:17.947352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766db85-c2d6-4bd3-92b5-1e165005f48c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:17.958376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 438/873] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:17 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:10:18.043160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:18.052357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766db85-c2d6-4bd3-92b5-1e165005f48c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:18.086595 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecf0ef-0c47-40a2-ae77-96d16d581b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:18.087197 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecf0ef-0c47-40a2-ae77-96d16d581b64 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:18.087475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecf0ef-0c47-40a2-ae77-96d16d581b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:18.087744 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecf0ef-0c47-40a2-ae77-96d16d581b64 None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:18.088428 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecf0ef-0c47-40a2-ae77-96d16d581b64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:18.088732 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecf0ef-0c47-40a2-ae77-96d16d581b64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:18.090175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecf0ef-0c47-40a2-ae77-96d16d581b64 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:18.103576 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 436/874] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:10:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:18.109570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3975bd-42da-4afd-aaa8-24443aa923dd None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:18.216703 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3975bd-42da-4afd-aaa8-24443aa923dd None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:18.218520 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:18.245056 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecf0ef-0c47-40a2-ae77-96d16d581b64 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:18.261709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e12154-2c99-4089-810a-032754ed39fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:18.262376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e12154-2c99-4089-810a-032754ed39fc None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:18.262656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e12154-2c99-4089-810a-032754ed39fc None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:18.262925 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e12154-2c99-4089-810a-032754ed39fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:18.263754 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e12154-2c99-4089-810a-032754ed39fc None admin] RBAC: Adding query filter params (role.id=60963384f3e54d9cbfbc8d12023a6cd9, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=bc69cf267bea41bf890bc7551d84ad6f) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:18.264061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e12154-2c99-4089-810a-032754ed39fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:18.266340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e12154-2c99-4089-810a-032754ed39fc None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:18.272968 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 439/875] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:10:18 2022] GET /identity/v3/role_assignments?user.id=bc69cf267bea41bf890bc7551d84ad6f&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=60963384f3e54d9cbfbc8d12023a6cd9 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:18.344864 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1e289-3777-451d-8c7d-97b2cef68bd1 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:18.345042 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1e289-3777-451d-8c7d-97b2cef68bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:18.345359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1e289-3777-451d-8c7d-97b2cef68bd1 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:18.348155 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 440/876] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:10:18.352854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a52662-75ea-4bf2-9ac1-522e4c7e6119 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:18.353194 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a52662-75ea-4bf2-9ac1-522e4c7e6119 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:18.353468 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a52662-75ea-4bf2-9ac1-522e4c7e6119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:18.368083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecf0ef-0c47-40a2-ae77-96d16d581b64 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:18.374361 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-31a52662-75ea-4bf2-9ac1-522e4c7e6119 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:18.376483 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-31a52662-75ea-4bf2-9ac1-522e4c7e6119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:18.412222 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37edd6-0437-49f8-b5c7-3ddf82b78b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:18.413210 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37edd6-0437-49f8-b5c7-3ddf82b78b1e None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:18.413807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37edd6-0437-49f8-b5c7-3ddf82b78b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:18.414083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37edd6-0437-49f8-b5c7-3ddf82b78b1e None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:18.414941 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37edd6-0437-49f8-b5c7-3ddf82b78b1e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:18.416437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37edd6-0437-49f8-b5c7-3ddf82b78b1e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:18.437655 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 437/877] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 12:10:18 2022] POST /identity/v3/services => generated 269 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:10:18.472522 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 441/878] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:10:18.738243 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37edd6-0437-49f8-b5c7-3ddf82b78b1e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:18.754803 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:18.845019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37edd6-0437-49f8-b5c7-3ddf82b78b1e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:18.880273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd7af7-df69-4814-b3b0-e3482d6d885b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:18.880913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd7af7-df69-4814-b3b0-e3482d6d885b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:18.881220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd7af7-df69-4814-b3b0-e3482d6d885b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:18.881495 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd7af7-df69-4814-b3b0-e3482d6d885b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:18.882076 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd7af7-df69-4814-b3b0-e3482d6d885b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:18.883438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd7af7-df69-4814-b3b0-e3482d6d885b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:18.893054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-74dd7af7-df69-4814-b3b0-e3482d6d885b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 12:10:18.893054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:18.893054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:18.893054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:18.893054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:18.893054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:18.893054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:18.893054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:18.893054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 12:10:18.893054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:18.893054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:18.893054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:18.893054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:18.893054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:18.893054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:18.893054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:18.893054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:18.894073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:18.894073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:18.894073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:18.894073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:18.894073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:18.894073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:18.894073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:18.894073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:18.894073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:18.894073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:18.894073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:18.894073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:18.894073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:18.894073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:18.894073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:18.894073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:18.894917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:18.894917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:18.894917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:18.894917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:18.894917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:18.894917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:18.894917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:18.894917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:18.894917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:18.894917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:18.894917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:18.894917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:18.894917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 12:10:18.894917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:18.898206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 438/879] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 12:10:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:18.900560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a52662-75ea-4bf2-9ac1-522e4c7e6119 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:18.993767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:19.002914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a52662-75ea-4bf2-9ac1-522e4c7e6119 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:19.042138 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b440d-df89-4913-9d28-08392516c84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:19.042771 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b440d-df89-4913-9d28-08392516c84b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:19.043049 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b440d-df89-4913-9d28-08392516c84b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:19.043317 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b440d-df89-4913-9d28-08392516c84b None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:19.043972 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b440d-df89-4913-9d28-08392516c84b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:19.044275 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b440d-df89-4913-9d28-08392516c84b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:19.045721 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b440d-df89-4913-9d28-08392516c84b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:19.056249 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 442/880] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 12:10:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:19.057730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd7af7-df69-4814-b3b0-e3482d6d885b None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:19.139974 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:19.150379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e915e5a-cf77-42c5-924f-e8e1f625bd80 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:19.151060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e915e5a-cf77-42c5-924f-e8e1f625bd80 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:19.151559 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e915e5a-cf77-42c5-924f-e8e1f625bd80 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:19.153105 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 443/881] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:10:19.159802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72b3d2-d476-446a-9342-feaacf8db900 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:19.160087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72b3d2-d476-446a-9342-feaacf8db900 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:19.160359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72b3d2-d476-446a-9342-feaacf8db900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:19.169593 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd7af7-df69-4814-b3b0-e3482d6d885b None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:19.182843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-eb72b3d2-d476-446a-9342-feaacf8db900 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:19.184545 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-eb72b3d2-d476-446a-9342-feaacf8db900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:19.206947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76205e9-eaf4-4c94-b672-fb343937d33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:19.207567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76205e9-eaf4-4c94-b672-fb343937d33b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:19.207847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76205e9-eaf4-4c94-b672-fb343937d33b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:19.208119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76205e9-eaf4-4c94-b672-fb343937d33b None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:19.208749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76205e9-eaf4-4c94-b672-fb343937d33b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:19.209048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76205e9-eaf4-4c94-b672-fb343937d33b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:19.210608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76205e9-eaf4-4c94-b672-fb343937d33b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:19.215901 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 439/882] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 12:10:19 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:19.271285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 444/883] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:19 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:19.544231 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76205e9-eaf4-4c94-b672-fb343937d33b None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:19.651840 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76205e9-eaf4-4c94-b672-fb343937d33b None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:19.686622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e0e52-2fce-4886-a01e-c1feeca253cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:19.688053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e0e52-2fce-4886-a01e-c1feeca253cd None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:19.688476 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e0e52-2fce-4886-a01e-c1feeca253cd None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:19.688885 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e0e52-2fce-4886-a01e-c1feeca253cd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:19.689637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e0e52-2fce-4886-a01e-c1feeca253cd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:19.691201 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e0e52-2fce-4886-a01e-c1feeca253cd None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:19.700511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-0e1e0e52-2fce-4886-a01e-c1feeca253cd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 12:10:19.700511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:19.700511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:19.700511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:19.700511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:19.700511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:19.700511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:19.700511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:19.700511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 12:10:19.700511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:19.700511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:19.700511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:19.700511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:19.700511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:19.700511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:19.700511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:19.700511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:19.701712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:19.701712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:19.701712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:19.701712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:19.701712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:19.701712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:19.701712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:19.701712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:19.701712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:19.701712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:19.701712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:19.701712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:19.701712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:19.701712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:19.701712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:19.701712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:19.703396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:19.703396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:19.703396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:19.703396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:19.703396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:19.703396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:19.703396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:19.703396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:19.703396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:19.703396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:19.703396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:19.703396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:19.703396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 12:10:19.703396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:19.705811 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 440/884] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:10:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:19.714493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72b3d2-d476-446a-9342-feaacf8db900 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:19.805217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:19.821485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72b3d2-d476-446a-9342-feaacf8db900 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:19.855731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d6e61-42b5-4c61-aaab-d94cc59e73ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:19.856556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d6e61-42b5-4c61-aaab-d94cc59e73ad None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:19.856975 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d6e61-42b5-4c61-aaab-d94cc59e73ad None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:19.857408 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d6e61-42b5-4c61-aaab-d94cc59e73ad None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:19.858188 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d6e61-42b5-4c61-aaab-d94cc59e73ad None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:19.859147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d6e61-42b5-4c61-aaab-d94cc59e73ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:19.861202 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d6e61-42b5-4c61-aaab-d94cc59e73ad None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:19.867571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 445/885] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:10:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:19.874508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e0e52-2fce-4886-a01e-c1feeca253cd None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:19.901645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:19.978865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e0e52-2fce-4886-a01e-c1feeca253cd None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:20.013824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67684f4-b6de-473e-8d76-2875a5d5e742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:20.014442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67684f4-b6de-473e-8d76-2875a5d5e742 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:20.014721 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67684f4-b6de-473e-8d76-2875a5d5e742 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:20.015599 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67684f4-b6de-473e-8d76-2875a5d5e742 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:20.016795 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67684f4-b6de-473e-8d76-2875a5d5e742 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:20.017102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67684f4-b6de-473e-8d76-2875a5d5e742 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:20.018496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67684f4-b6de-473e-8d76-2875a5d5e742 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:20.024022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 441/886] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:10:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:20.030387 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d6e61-42b5-4c61-aaab-d94cc59e73ad None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:20.141390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d6e61-42b5-4c61-aaab-d94cc59e73ad None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:20.178224 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2dfbe8-c9fb-4f81-93d3-a5e3788cccd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:20.178839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2dfbe8-c9fb-4f81-93d3-a5e3788cccd9 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:20.179118 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2dfbe8-c9fb-4f81-93d3-a5e3788cccd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:20.179390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2dfbe8-c9fb-4f81-93d3-a5e3788cccd9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:20.182643 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2dfbe8-c9fb-4f81-93d3-a5e3788cccd9 None admin] RBAC: Adding query filter params (interface=public, service_id=27c0ee91434240ee948b3f8ade4bee62, region_id=RegionOne) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:20.182643 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2dfbe8-c9fb-4f81-93d3-a5e3788cccd9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:20.182643 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2dfbe8-c9fb-4f81-93d3-a5e3788cccd9 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:20.190525 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 446/887] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Sep 28 12:10:20 2022] GET /identity/v3/endpoints?service_id=27c0ee91434240ee948b3f8ade4bee62&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:20.195956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67684f4-b6de-473e-8d76-2875a5d5e742 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:20.269152 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:20.306303 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67684f4-b6de-473e-8d76-2875a5d5e742 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:20.341693 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53507e96-f01b-4531-ad5b-d75998703816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:20.342306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53507e96-f01b-4531-ad5b-d75998703816 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:20.342584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53507e96-f01b-4531-ad5b-d75998703816 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:20.342858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53507e96-f01b-4531-ad5b-d75998703816 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:20.344175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53507e96-f01b-4531-ad5b-d75998703816 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:20.345564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53507e96-f01b-4531-ad5b-d75998703816 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:20.351808 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 442/888] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 12:10:20 2022] GET /identity/v3/services => generated 1268 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:10:20.636734 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca5e69-a977-41aa-9101-5b64484beceb None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:20.637471 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca5e69-a977-41aa-9101-5b64484beceb None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:20.637968 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca5e69-a977-41aa-9101-5b64484beceb None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:20.639738 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 447/889] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:20.645851 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed113-09db-4105-b07d-295b71e8d8f7 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:20.646755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed113-09db-4105-b07d-295b71e8d8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:20.647233 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed113-09db-4105-b07d-295b71e8d8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:20.666429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-1f9ed113-09db-4105-b07d-295b71e8d8f7 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:20.667953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1f9ed113-09db-4105-b07d-295b71e8d8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:20.677121 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dbeef-af35-4b0b-aa3a-a401d6046b68 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:20.677712 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dbeef-af35-4b0b-aa3a-a401d6046b68 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:20.678044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dbeef-af35-4b0b-aa3a-a401d6046b68 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:20.679802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 448/890] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:10:20.684717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd4d85-4387-4b10-bc0a-80af5f1be5c6 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:20.685296 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd4d85-4387-4b10-bc0a-80af5f1be5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:20.685791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd4d85-4387-4b10-bc0a-80af5f1be5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:20.706010 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-64bd4d85-4387-4b10-bc0a-80af5f1be5c6 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:20.707568 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-64bd4d85-4387-4b10-bc0a-80af5f1be5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:20.717883 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:20.753072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 443/891] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:20.793445 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 449/892] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:10:21.025123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ed113-09db-4105-b07d-295b71e8d8f7 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:21.067090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd4d85-4387-4b10-bc0a-80af5f1be5c6 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:21.095889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:21.147022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ed113-09db-4105-b07d-295b71e8d8f7 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:21.200517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c278ac-83c1-4f3c-8599-d35e07c8f810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:21.201426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c278ac-83c1-4f3c-8599-d35e07c8f810 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:21.201706 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c278ac-83c1-4f3c-8599-d35e07c8f810 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:21.201977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c278ac-83c1-4f3c-8599-d35e07c8f810 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:21.209240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd4d85-4387-4b10-bc0a-80af5f1be5c6 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:21.215430 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c278ac-83c1-4f3c-8599-d35e07c8f810 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:10:21.223251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-87c278ac-83c1-4f3c-8599-d35e07c8f810 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 12:10:21.223251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:21.223251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:21.223251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:21.223251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:21.223251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:21.223251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:21.223251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:21.223251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:21.223251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:21.223251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:21.223251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:21.223251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:10:21.223251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:10:21.223251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:10:21.223251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:10:21.224249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:10:21.224249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:10:21.224249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:10:21.224249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:10:21.224249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:21.224249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:21.224249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:21.224249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:21.224249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:21.224249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:21.224249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:21.224249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:21.224249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:21.224249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:21.224249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:21.224249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:21.225165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:21.225165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:21.225165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:21.225165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:21.225165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:10:21.225165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:10:21.225165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:10:21.225165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:10:21.225165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:10:21.225165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:10:21.225165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 12:10:21.225165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:21.229540 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 444/893] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Sep 28 12:10:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:21.230560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c278ac-83c1-4f3c-8599-d35e07c8f810 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:21.259351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822cd601-c00e-427a-ad05-27492b3f171e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:21.260394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cd601-c00e-427a-ad05-27492b3f171e None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:21.260894 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cd601-c00e-427a-ad05-27492b3f171e None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:21.261939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cd601-c00e-427a-ad05-27492b3f171e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:21.263039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822cd601-c00e-427a-ad05-27492b3f171e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:21.264821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822cd601-c00e-427a-ad05-27492b3f171e None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:21.278490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-822cd601-c00e-427a-ad05-27492b3f171e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 12:10:21.278490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:21.278490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:21.278490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:21.278490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:21.278490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:21.278490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:21.278490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:21.278490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 12:10:21.278490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:21.278490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:21.278490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:21.278490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:21.278490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:21.278490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:21.278490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:21.279504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:21.279504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:21.279504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:21.279504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:21.279504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:21.279504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:21.279504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:21.279504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:21.279504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:21.279504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:21.279504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:21.279504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:21.279504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:21.279504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:21.279504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:21.279504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:21.279504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:21.280410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:21.280410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:21.280410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:21.280410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:21.280410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:21.280410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:21.280410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:21.280410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:21.280410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:21.280410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:21.280410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:21.280410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:21.280410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 12:10:21.280410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:21.280410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 450/894] 10.222.0.60 () {56 vars in 1132 bytes} [Wed Sep 28 12:10:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:21.288250 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822cd601-c00e-427a-ad05-27492b3f171e None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:21.299984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:21.359948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c278ac-83c1-4f3c-8599-d35e07c8f810 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:21.384905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:21.394775 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822cd601-c00e-427a-ad05-27492b3f171e None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:21.399829 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8d233-3e0b-4cf5-b177-3aec6041aa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:21.400733 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8d233-3e0b-4cf5-b177-3aec6041aa42 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:21.401279 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8d233-3e0b-4cf5-b177-3aec6041aa42 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:21.401760 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8d233-3e0b-4cf5-b177-3aec6041aa42 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:21.402643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8d233-3e0b-4cf5-b177-3aec6041aa42 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:21.403180 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8d233-3e0b-4cf5-b177-3aec6041aa42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:21.404811 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8d233-3e0b-4cf5-b177-3aec6041aa42 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:21.415244 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 445/895] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Sep 28 12:10:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:21.421062 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8d233-3e0b-4cf5-b177-3aec6041aa42 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:21.441732 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aaa257-b099-4fd9-a1f1-2d4dcf667b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:21.442389 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aaa257-b099-4fd9-a1f1-2d4dcf667b90 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:21.442950 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aaa257-b099-4fd9-a1f1-2d4dcf667b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:21.443233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aaa257-b099-4fd9-a1f1-2d4dcf667b90 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:21.443879 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aaa257-b099-4fd9-a1f1-2d4dcf667b90 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:21.444184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aaa257-b099-4fd9-a1f1-2d4dcf667b90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:21.445892 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aaa257-b099-4fd9-a1f1-2d4dcf667b90 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:21.457199 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 451/896] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 12:10:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:21.459280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aaa257-b099-4fd9-a1f1-2d4dcf667b90 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:21.551328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:21.557111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8d233-3e0b-4cf5-b177-3aec6041aa42 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:21.585892 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aaa257-b099-4fd9-a1f1-2d4dcf667b90 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:21.591802 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccab419-9088-49c6-9881-ac18fa1f5662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:21.592443 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccab419-9088-49c6-9881-ac18fa1f5662 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:21.592721 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccab419-9088-49c6-9881-ac18fa1f5662 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:21.593106 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccab419-9088-49c6-9881-ac18fa1f5662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:21.604388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccab419-9088-49c6-9881-ac18fa1f5662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:21.605853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccab419-9088-49c6-9881-ac18fa1f5662 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:21.621725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-4ccab419-9088-49c6-9881-ac18fa1f5662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:21.621725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:21.621725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:21.621725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:21.621725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:21.621725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:21.621725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:21.621725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:21.621725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:21.621725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:21.621725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:21.621725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:21.621725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:21.621725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:21.621725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:21.621725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:21.622730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:21.622730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:21.622730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:21.622730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:21.622730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:21.622730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:21.622730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:21.622730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:21.622730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:21.622730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:21.622730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:21.622730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:21.622730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:21.622730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:21.622730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:21.622730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:21.622730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:21.628383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:21.628383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:21.628383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:21.628595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45de63-0ca0-4b2c-965f-a564751a24cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:21.628595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45de63-0ca0-4b2c-965f-a564751a24cb None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:21.628595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45de63-0ca0-4b2c-965f-a564751a24cb None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:21.628595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45de63-0ca0-4b2c-965f-a564751a24cb None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:21.628886 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 446/897] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:21.628970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45de63-0ca0-4b2c-965f-a564751a24cb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:21.628970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45de63-0ca0-4b2c-965f-a564751a24cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:21.629104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ab681-9011-4f4a-93a7-8ed526d2422f None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:21.629104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ab681-9011-4f4a-93a7-8ed526d2422f None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:21.629104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ab681-9011-4f4a-93a7-8ed526d2422f None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:21.629326 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45de63-0ca0-4b2c-965f-a564751a24cb None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:21.629405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 447/898] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:21.629405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ab681-9011-4f4a-93a7-8ed526d2422f None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:21.641213 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 452/899] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 12:10:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:21.643354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fa0bf-c438-42ac-b09d-8bb6b75305d2 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:21.643694 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fa0bf-c438-42ac-b09d-8bb6b75305d2 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:21.643990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fa0bf-c438-42ac-b09d-8bb6b75305d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:21.662631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-f27fa0bf-c438-42ac-b09d-8bb6b75305d2 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:21.663813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-f27fa0bf-c438-42ac-b09d-8bb6b75305d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:21.669668 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:21.733932 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 453/900] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:21.734535 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fa0bf-c438-42ac-b09d-8bb6b75305d2 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:21.752218 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ab681-9011-4f4a-93a7-8ed526d2422f None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:21.791932 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfef420-214e-45dc-b65d-60b712ab87ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:21.793983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfef420-214e-45dc-b65d-60b712ab87ca None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:21.794272 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfef420-214e-45dc-b65d-60b712ab87ca None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:21.794549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfef420-214e-45dc-b65d-60b712ab87ca None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:21.795198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfef420-214e-45dc-b65d-60b712ab87ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:21.795496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfef420-214e-45dc-b65d-60b712ab87ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:21.797367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfef420-214e-45dc-b65d-60b712ab87ca None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:21.813580 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 448/901] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:21.817893 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfef420-214e-45dc-b65d-60b712ab87ca None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:21.819794 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:21.861762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fa0bf-c438-42ac-b09d-8bb6b75305d2 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:21.901255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87ef05-7fda-42dd-ab1c-bc5c7b91e43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:21.901856 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87ef05-7fda-42dd-ab1c-bc5c7b91e43c None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:21.902131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87ef05-7fda-42dd-ab1c-bc5c7b91e43c None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:21.902399 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87ef05-7fda-42dd-ab1c-bc5c7b91e43c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:21.903086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87ef05-7fda-42dd-ab1c-bc5c7b91e43c None admin] RBAC: Adding query filter params (interface=public, service_id=a4f8b788f54d41f6a87afdf3dd96f2fb, region_id=RegionOne) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:21.903388 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87ef05-7fda-42dd-ab1c-bc5c7b91e43c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:21.904853 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87ef05-7fda-42dd-ab1c-bc5c7b91e43c None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:21.911169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 454/902] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Sep 28 12:10:21 2022] GET /identity/v3/endpoints?service_id=a4f8b788f54d41f6a87afdf3dd96f2fb&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:21.916710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87ef05-7fda-42dd-ab1c-bc5c7b91e43c None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:21.933363 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:21.937675 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfef420-214e-45dc-b65d-60b712ab87ca None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:21.977335 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda0211-6c69-410f-8ed0-32881d2d931c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:21.978913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda0211-6c69-410f-8ed0-32881d2d931c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:21.979423 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda0211-6c69-410f-8ed0-32881d2d931c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:21.980143 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda0211-6c69-410f-8ed0-32881d2d931c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:21.990636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda0211-6c69-410f-8ed0-32881d2d931c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:21.992856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda0211-6c69-410f-8ed0-32881d2d931c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:22.008211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-1bda0211-6c69-410f-8ed0-32881d2d931c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 12:10:22.008211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:22.008211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:22.008211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:22.008211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:22.008211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:22.008211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:22.008211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:22.008211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:22.008211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:22.008211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:22.008211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:22.008211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:10:22.008211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:10:22.008211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:10:22.008211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:22.009223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:22.010485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:22.010485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:22.010485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:10:22.010485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:10:22.010485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:10:22.010485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:10:22.010485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:10:22.010485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:10:22.010485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 12:10:22.010485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:22.010485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 449/903] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 12:10:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:22.018162 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda0211-6c69-410f-8ed0-32881d2d931c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:22.045573 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87ef05-7fda-42dd-ab1c-bc5c7b91e43c None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:22.081999 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:22.088126 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1431b-65f9-44a5-9f5c-bb96302933e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:22.088807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1431b-65f9-44a5-9f5c-bb96302933e3 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:22.088992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1431b-65f9-44a5-9f5c-bb96302933e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:22.089287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1431b-65f9-44a5-9f5c-bb96302933e3 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:22.089876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1431b-65f9-44a5-9f5c-bb96302933e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:22.091227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1431b-65f9-44a5-9f5c-bb96302933e3 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:22.098057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 455/904] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 12:10:21 2022] GET /identity/v3/services => generated 1268 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:10:22.102053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1431b-65f9-44a5-9f5c-bb96302933e3 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:22.176961 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda0211-6c69-410f-8ed0-32881d2d931c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:22.228957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1431b-65f9-44a5-9f5c-bb96302933e3 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:22.229849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b66a0b-ce3d-4ed1-970b-f87258bf3254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:22.230478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b66a0b-ce3d-4ed1-970b-f87258bf3254 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:22.230755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b66a0b-ce3d-4ed1-970b-f87258bf3254 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:22.231026 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b66a0b-ce3d-4ed1-970b-f87258bf3254 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:22.231775 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b66a0b-ce3d-4ed1-970b-f87258bf3254 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:22.232081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b66a0b-ce3d-4ed1-970b-f87258bf3254 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:22.233641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b66a0b-ce3d-4ed1-970b-f87258bf3254 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:22.260520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 450/905] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Sep 28 12:10:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:22.261521 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b66a0b-ce3d-4ed1-970b-f87258bf3254 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:22.279622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:22.284604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018c4a2-98ae-45ea-9e13-0ac2e8c03065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:22.287843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018c4a2-98ae-45ea-9e13-0ac2e8c03065 None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:22.287843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018c4a2-98ae-45ea-9e13-0ac2e8c03065 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:22.287843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018c4a2-98ae-45ea-9e13-0ac2e8c03065 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:22.287843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018c4a2-98ae-45ea-9e13-0ac2e8c03065 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:22.288306 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018c4a2-98ae-45ea-9e13-0ac2e8c03065 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:22.310060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 456/906] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 12:10:22 2022] POST /identity/v3/services => generated 241 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:10:22.391326 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b66a0b-ce3d-4ed1-970b-f87258bf3254 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:22.436383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba9523-49f7-48bc-b812-faf0f9c2c5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:22.437623 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba9523-49f7-48bc-b812-faf0f9c2c5c9 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:22.437909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba9523-49f7-48bc-b812-faf0f9c2c5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:22.438173 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba9523-49f7-48bc-b812-faf0f9c2c5c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:22.525092 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba9523-49f7-48bc-b812-faf0f9c2c5c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:22.526566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba9523-49f7-48bc-b812-faf0f9c2c5c9 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:22.539394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:22.602915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42542ab5-c347-49a5-b795-625b52c31f82 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:22.603256 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42542ab5-c347-49a5-b795-625b52c31f82 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:22.603474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42542ab5-c347-49a5-b795-625b52c31f82 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:22.605037 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 457/907] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:22.610964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f599263-f347-48c1-a322-5dab11732f67 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:22.611248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f599263-f347-48c1-a322-5dab11732f67 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:22.611522 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f599263-f347-48c1-a322-5dab11732f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:22.633665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-9f599263-f347-48c1-a322-5dab11732f67 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:22.634878 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9f599263-f347-48c1-a322-5dab11732f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:22.636212 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 451/908] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:10:22 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:10:22.642049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba9523-49f7-48bc-b812-faf0f9c2c5c9 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:22.697366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:22.717325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 458/909] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:10:22.753621 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba9523-49f7-48bc-b812-faf0f9c2c5c9 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:22.764596 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:22.789002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f822a4a-c88a-4db0-ac36-56b0bd912917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:22.789706 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f822a4a-c88a-4db0-ac36-56b0bd912917 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:22.789987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f822a4a-c88a-4db0-ac36-56b0bd912917 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:22.790256 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f822a4a-c88a-4db0-ac36-56b0bd912917 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:22.801295 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f822a4a-c88a-4db0-ac36-56b0bd912917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:22.802779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f822a4a-c88a-4db0-ac36-56b0bd912917 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:22.818527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-0f822a4a-c88a-4db0-ac36-56b0bd912917 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:22.818527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:22.818527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:22.818527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:22.818527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:22.818527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:22.818527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:22.818527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:22.818527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:22.818527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:22.818527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:22.818527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:22.818527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:22.818527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:22.818527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:22.818527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:22.819513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:22.819513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:22.819513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:22.819513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:22.819513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:22.819513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:22.819513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:22.819513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:22.819513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:22.819513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:22.819513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:22.819513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:22.819513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:22.819513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:22.819513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:22.819513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:22.819513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:22.820417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:22.820417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:22.820417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:22.820417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 452/910] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:22.825569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f599263-f347-48c1-a322-5dab11732f67 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:22.929195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f599263-f347-48c1-a322-5dab11732f67 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:22.968315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bc1bb-a2fe-450e-b26d-761ad7a2bd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:22.968934 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bc1bb-a2fe-450e-b26d-761ad7a2bd62 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:22.969242 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bc1bb-a2fe-450e-b26d-761ad7a2bd62 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:22.969591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577bc1bb-a2fe-450e-b26d-761ad7a2bd62 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:22.970209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bc1bb-a2fe-450e-b26d-761ad7a2bd62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:22.970514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bc1bb-a2fe-450e-b26d-761ad7a2bd62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:22.971877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577bc1bb-a2fe-450e-b26d-761ad7a2bd62 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:22.984308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 459/911] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:22.989797 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f822a4a-c88a-4db0-ac36-56b0bd912917 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:22.996035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bc1bb-a2fe-450e-b26d-761ad7a2bd62 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:23.043585 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:23.105409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577bc1bb-a2fe-450e-b26d-761ad7a2bd62 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:23.109285 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f822a4a-c88a-4db0-ac36-56b0bd912917 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:23.142958 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484a507-8764-4d04-b235-3825cec24d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:23.143597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484a507-8764-4d04-b235-3825cec24d5a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:23.143880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484a507-8764-4d04-b235-3825cec24d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:23.144155 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484a507-8764-4d04-b235-3825cec24d5a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:23.144733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484a507-8764-4d04-b235-3825cec24d5a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:23.146134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484a507-8764-4d04-b235-3825cec24d5a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:23.147268 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aac428-1671-4662-ad48-31612ac06703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:23.148183 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aac428-1671-4662-ad48-31612ac06703 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:23.148721 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aac428-1671-4662-ad48-31612ac06703 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:23.149319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aac428-1671-4662-ad48-31612ac06703 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:23.156339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-6484a507-8764-4d04-b235-3825cec24d5a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 12:10:23.156339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:23.156339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:23.156339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:23.156339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:23.156339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:23.156339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:23.156339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:23.156339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 12:10:23.156339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:23.156339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:23.156339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:23.156339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:23.156339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:23.156339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:23.156339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:23.156339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:23.157969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:23.157969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:23.157969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:23.157969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:23.157969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:23.157969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:23.157969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:23.157969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:23.157969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:23.157969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:23.157969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:23.157969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:23.157969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:23.157969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:23.157969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:23.157969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:23.158992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:23.158992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:23.158992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:23.158992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:23.158992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:23.158992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:23.158992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:23.158992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:23.158992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:23.158992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:23.158992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:23.158992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:23.158992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 12:10:23.158992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:23.160641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aac428-1671-4662-ad48-31612ac06703 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:23.162405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aac428-1671-4662-ad48-31612ac06703 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:23.164621 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 460/912] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:10:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:23.168751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484a507-8764-4d04-b235-3825cec24d5a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:23.182955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-c5aac428-1671-4662-ad48-31612ac06703 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:23.182955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:23.182955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:23.182955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:23.182955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:23.182955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:23.182955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:23.182955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:23.182955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:23.182955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:23.182955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:23.182955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:23.182955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:10:23.182955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:10:23.182955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:10:23.183870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:10:23.183870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:23.183870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:23.183870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:23.183870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:23.183870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:23.183870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:23.183870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:23.183870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:23.183870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:23.183870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:23.183870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:23.183870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:23.183870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:23.183870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:23.183870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:23.183870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:23.184748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:10:23.184748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:10:23.184748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:10:23.184748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:10:23.184748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:10:23.184748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:10:23.184748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:23.184748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:23.191584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 453/913] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:23.192726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aac428-1671-4662-ad48-31612ac06703 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:23.220180 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:23.283513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484a507-8764-4d04-b235-3825cec24d5a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:23.294737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:23.304503 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aac428-1671-4662-ad48-31612ac06703 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:23.324612 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d7cb5-e0e1-422d-be9f-c41a53a5a522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:23.325251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d7cb5-e0e1-422d-be9f-c41a53a5a522 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:23.325530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d7cb5-e0e1-422d-be9f-c41a53a5a522 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:23.325804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d7cb5-e0e1-422d-be9f-c41a53a5a522 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:23.326429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d7cb5-e0e1-422d-be9f-c41a53a5a522 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:23.326792 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d7cb5-e0e1-422d-be9f-c41a53a5a522 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:23.329256 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d7cb5-e0e1-422d-be9f-c41a53a5a522 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:23.338915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 461/914] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:10:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:23.339698 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d7cb5-e0e1-422d-be9f-c41a53a5a522 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:23.345357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfaadac-c10c-4c49-b410-fb05d056c40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:23.346220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfaadac-c10c-4c49-b410-fb05d056c40e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:23.346694 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfaadac-c10c-4c49-b410-fb05d056c40e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:23.347156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfaadac-c10c-4c49-b410-fb05d056c40e None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:23.348033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfaadac-c10c-4c49-b410-fb05d056c40e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:23.348668 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfaadac-c10c-4c49-b410-fb05d056c40e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:23.350485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfaadac-c10c-4c49-b410-fb05d056c40e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:23.370126 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 454/915] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:10:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:23.371576 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfaadac-c10c-4c49-b410-fb05d056c40e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:23.451163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d7cb5-e0e1-422d-be9f-c41a53a5a522 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:23.473059 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:23.493836 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d54d4-d83b-4994-af88-277bc09a75b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:23.494593 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d54d4-d83b-4994-af88-277bc09a75b3 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:23.494949 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d54d4-d83b-4994-af88-277bc09a75b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:23.495318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d54d4-d83b-4994-af88-277bc09a75b3 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:23.496045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d54d4-d83b-4994-af88-277bc09a75b3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:23.496441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d54d4-d83b-4994-af88-277bc09a75b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:23.497254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfaadac-c10c-4c49-b410-fb05d056c40e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:23.498045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d54d4-d83b-4994-af88-277bc09a75b3 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:23.505175 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 462/916] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Sep 28 12:10:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:23.510376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d54d4-d83b-4994-af88-277bc09a75b3 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:23.542413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275e5e05-ad13-4e42-8788-7cfdf46082f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:23.543103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e5e05-ad13-4e42-8788-7cfdf46082f0 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:23.543427 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e5e05-ad13-4e42-8788-7cfdf46082f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:23.543650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e5e05-ad13-4e42-8788-7cfdf46082f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:23.544425 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e5e05-ad13-4e42-8788-7cfdf46082f0 None admin] RBAC: Adding query filter params (role.id=8af05c40a1604332a08a2c6ba7ad6de7, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=bc69cf267bea41bf890bc7551d84ad6f) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:23.544737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e5e05-ad13-4e42-8788-7cfdf46082f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:23.547137 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275e5e05-ad13-4e42-8788-7cfdf46082f0 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:23.551994 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:23.554273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 455/917] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:10:23 2022] GET /identity/v3/role_assignments?user.id=bc69cf267bea41bf890bc7551d84ad6f&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=8af05c40a1604332a08a2c6ba7ad6de7 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:23.625422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d54d4-d83b-4994-af88-277bc09a75b3 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:23.662675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ba67a-3de3-4f62-881c-0acdc05b321c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:23.663296 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ba67a-3de3-4f62-881c-0acdc05b321c None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:23.663569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ba67a-3de3-4f62-881c-0acdc05b321c None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:23.663835 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ba67a-3de3-4f62-881c-0acdc05b321c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:23.664499 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ba67a-3de3-4f62-881c-0acdc05b321c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:23.665893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ba67a-3de3-4f62-881c-0acdc05b321c None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:23.690481 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 463/918] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:10:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:10:24.387478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2fc82-b6a2-477d-b2cc-83ad524642e0 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:24.387991 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2fc82-b6a2-477d-b2cc-83ad524642e0 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:24.387991 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2fc82-b6a2-477d-b2cc-83ad524642e0 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:24.389637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 456/919] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:24.395556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789982f-1c38-410c-b791-ac1ff263a525 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:24.395998 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789982f-1c38-410c-b791-ac1ff263a525 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:24.396073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789982f-1c38-410c-b791-ac1ff263a525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:24.414786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-0789982f-1c38-410c-b791-ac1ff263a525 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:24.415960 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-0789982f-1c38-410c-b791-ac1ff263a525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:24.490206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 464/920] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:10:24.670740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f97c6a-c7fa-43bd-a81f-d0f95d1c7582 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:24.671692 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f97c6a-c7fa-43bd-a81f-d0f95d1c7582 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:24.672030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f97c6a-c7fa-43bd-a81f-d0f95d1c7582 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:24.673734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 457/921] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:24.679371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdc212-00bb-48f9-bfd1-8d715414d83a None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:24.679660 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdc212-00bb-48f9-bfd1-8d715414d83a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:24.679936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdc212-00bb-48f9-bfd1-8d715414d83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:24.698172 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-b7fdc212-00bb-48f9-bfd1-8d715414d83a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:24.699346 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b7fdc212-00bb-48f9-bfd1-8d715414d83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:24.712520 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:24.770054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 465/922] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:24.771235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f97c6a-c7fa-43bd-a81f-d0f95d1c7582 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:24.885750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f97c6a-c7fa-43bd-a81f-d0f95d1c7582 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:24.923489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63c478-1b66-40d0-a066-93c22f634a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:24.924474 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63c478-1b66-40d0-a066-93c22f634a3c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:24.924972 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63c478-1b66-40d0-a066-93c22f634a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:24.925462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63c478-1b66-40d0-a066-93c22f634a3c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:24.926244 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63c478-1b66-40d0-a066-93c22f634a3c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:24.927817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63c478-1b66-40d0-a066-93c22f634a3c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:24.937413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-ba63c478-1b66-40d0-a066-93c22f634a3c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 12:10:24.937413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:24.937413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:24.937413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:24.937413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:24.937413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:24.937413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:24.937413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:24.937413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 12:10:24.937413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:24.937413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:24.937413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:24.937413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:24.937413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:24.937413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:24.937413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:24.938348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:24.938348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:24.938348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:24.938348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:24.938348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:24.938348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:24.938348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:24.938348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:24.938348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:24.938348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:24.938348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:24.938348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:24.938348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:24.938348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:24.938348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:24.938348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:24.938348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:24.939270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:24.939270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:24.939270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:24.939270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:24.939270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:24.939270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:24.939270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:24.939270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:24.939270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:24.939270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:24.939270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:24.939270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:24.939270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 12:10:24.939270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:24.941994 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 458/923] 10.222.0.60 () {56 vars in 1132 bytes} [Wed Sep 28 12:10:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:24.948170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fdc212-00bb-48f9-bfd1-8d715414d83a None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:25.036049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63c478-1b66-40d0-a066-93c22f634a3c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:25.050487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:25.056476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fdc212-00bb-48f9-bfd1-8d715414d83a None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:25.093577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca8967-f019-4373-a0f2-9e666ff631a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:25.094255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca8967-f019-4373-a0f2-9e666ff631a5 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:25.094534 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca8967-f019-4373-a0f2-9e666ff631a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:25.094980 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca8967-f019-4373-a0f2-9e666ff631a5 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:25.095974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca8967-f019-4373-a0f2-9e666ff631a5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:25.096285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca8967-f019-4373-a0f2-9e666ff631a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:25.097964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca8967-f019-4373-a0f2-9e666ff631a5 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:25.103073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 466/924] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 12:10:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:25.109572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca8967-f019-4373-a0f2-9e666ff631a5 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:25.125663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:25.154035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63c478-1b66-40d0-a066-93c22f634a3c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:25.191382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c59251-cd2d-4c4a-86b1-b9a33023e484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:25.192019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c59251-cd2d-4c4a-86b1-b9a33023e484 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:25.192292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c59251-cd2d-4c4a-86b1-b9a33023e484 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:25.192612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c59251-cd2d-4c4a-86b1-b9a33023e484 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:25.193288 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c59251-cd2d-4c4a-86b1-b9a33023e484 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:25.194693 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c59251-cd2d-4c4a-86b1-b9a33023e484 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:25.204150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-b7c59251-cd2d-4c4a-86b1-b9a33023e484 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 12:10:25.204150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:25.204150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:25.204150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:25.204150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:25.204150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:25.204150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:25.204150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:25.204150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 12:10:25.204150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:25.204150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:25.204150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:25.204150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:25.204150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:25.204150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:25.204150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:25.204150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:25.205419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:25.205419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:25.205419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:25.205419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:25.205419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:25.205419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:25.205419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:25.205419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:25.205419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:25.205419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:25.205419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:25.205419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:25.205419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:25.205419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:25.205419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:25.205419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:25.206473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:25.206473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:25.206473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:25.206473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:25.206473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:25.206473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:25.206473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:25.206473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:25.206473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:25.206473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:25.206473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:25.206473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:25.206473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 12:10:25.206473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:25.206473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 459/925] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 12:10:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:25.213735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c59251-cd2d-4c4a-86b1-b9a33023e484 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:25.223870 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca8967-f019-4373-a0f2-9e666ff631a5 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:25.260918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2c92b-2ab0-4479-bc1d-3c2dbaf425d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:25.261661 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2c92b-2ab0-4479-bc1d-3c2dbaf425d5 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:25.261949 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2c92b-2ab0-4479-bc1d-3c2dbaf425d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:25.262226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2c92b-2ab0-4479-bc1d-3c2dbaf425d5 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:25.262846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2c92b-2ab0-4479-bc1d-3c2dbaf425d5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:25.263156 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2c92b-2ab0-4479-bc1d-3c2dbaf425d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:25.264631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2c92b-2ab0-4479-bc1d-3c2dbaf425d5 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:25.275303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 467/926] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 12:10:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:25.276680 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2c92b-2ab0-4479-bc1d-3c2dbaf425d5 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:25.292321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:25.320380 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c59251-cd2d-4c4a-86b1-b9a33023e484 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:25.354803 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11865a-6d5c-4efc-99c7-3fc1e7813de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:25.355502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11865a-6d5c-4efc-99c7-3fc1e7813de0 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:25.355849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11865a-6d5c-4efc-99c7-3fc1e7813de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:25.356186 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11865a-6d5c-4efc-99c7-3fc1e7813de0 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:25.356926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11865a-6d5c-4efc-99c7-3fc1e7813de0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:25.357323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11865a-6d5c-4efc-99c7-3fc1e7813de0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:25.359552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11865a-6d5c-4efc-99c7-3fc1e7813de0 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:25.364152 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:25.369458 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 460/927] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 12:10:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:25.371052 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11865a-6d5c-4efc-99c7-3fc1e7813de0 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:25.390893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2c92b-2ab0-4479-bc1d-3c2dbaf425d5 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:25.429345 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6eed52-cbe8-4520-aed2-c7c5768570f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:25.430128 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6eed52-cbe8-4520-aed2-c7c5768570f6 None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:25.430309 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6eed52-cbe8-4520-aed2-c7c5768570f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:25.433178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6eed52-cbe8-4520-aed2-c7c5768570f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:25.433178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6eed52-cbe8-4520-aed2-c7c5768570f6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:25.433178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6eed52-cbe8-4520-aed2-c7c5768570f6 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:25.464089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 468/928] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:10:25 2022] POST /identity/v3/endpoints => generated 347 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:10:25.493363 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11865a-6d5c-4efc-99c7-3fc1e7813de0 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:25.542274 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80a865-ce9b-4f94-8d39-2396326173fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:25.542948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80a865-ce9b-4f94-8d39-2396326173fb None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:25.543179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80a865-ce9b-4f94-8d39-2396326173fb None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:25.543455 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80a865-ce9b-4f94-8d39-2396326173fb None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:25.544098 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80a865-ce9b-4f94-8d39-2396326173fb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:25.544453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80a865-ce9b-4f94-8d39-2396326173fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:25.545905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80a865-ce9b-4f94-8d39-2396326173fb None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:25.548456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:25.553656 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 461/929] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 12:10:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:25.559326 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6eed52-cbe8-4520-aed2-c7c5768570f6 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:25.682430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6eed52-cbe8-4520-aed2-c7c5768570f6 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:25.713808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:25.720283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05824cf-84d0-451c-b437-79f4065a8046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:25.720949 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05824cf-84d0-451c-b437-79f4065a8046 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:25.721268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05824cf-84d0-451c-b437-79f4065a8046 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:25.721540 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05824cf-84d0-451c-b437-79f4065a8046 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:25.722248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05824cf-84d0-451c-b437-79f4065a8046 None admin] RBAC: Adding query filter params (interface=public, service_id=bb5e935613cd46f38c3dadefdfe185c1, region_id=RegionOne) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:25.722744 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05824cf-84d0-451c-b437-79f4065a8046 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:25.724418 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05824cf-84d0-451c-b437-79f4065a8046 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:25.730600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 469/930] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Sep 28 12:10:25 2022] GET /identity/v3/endpoints?service_id=bb5e935613cd46f38c3dadefdfe185c1&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:25.736949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80a865-ce9b-4f94-8d39-2396326173fb None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:25.849029 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80a865-ce9b-4f94-8d39-2396326173fb None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:25.885922 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e65a3-d6fe-4670-aeca-24c3081e2171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:25.886828 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e65a3-d6fe-4670-aeca-24c3081e2171 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:25.887307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e65a3-d6fe-4670-aeca-24c3081e2171 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:25.887771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e65a3-d6fe-4670-aeca-24c3081e2171 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:25.888554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e65a3-d6fe-4670-aeca-24c3081e2171 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:25.890731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e65a3-d6fe-4670-aeca-24c3081e2171 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:25.898134 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 462/931] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 12:10:25 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:10:25.931726 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed45d49-6f7a-4373-aeaa-909b608b2e3c None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:25.932037 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed45d49-6f7a-4373-aeaa-909b608b2e3c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:25.932426 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed45d49-6f7a-4373-aeaa-909b608b2e3c None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:25.934295 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 470/932] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:25.940231 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9a58a-a19d-4860-a0e8-c6971789b861 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:25.940518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9a58a-a19d-4860-a0e8-c6971789b861 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:25.940933 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9a58a-a19d-4860-a0e8-c6971789b861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:25.961615 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-73a9a58a-a19d-4860-a0e8-c6971789b861 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:25.963766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-73a9a58a-a19d-4860-a0e8-c6971789b861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:26.028384 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dcd5d-d652-418a-819f-87369c9dc3b9 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:26.028739 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dcd5d-d652-418a-819f-87369c9dc3b9 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:26.028927 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6dcd5d-d652-418a-819f-87369c9dc3b9 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:26.031273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 471/933] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:26.035757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:26.036905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2307b-5879-4913-a891-3c918b749759 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:26.037218 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2307b-5879-4913-a891-3c918b749759 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:26.037491 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2307b-5879-4913-a891-3c918b749759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:26.053002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 463/934] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:26.061629 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-1cd2307b-5879-4913-a891-3c918b749759 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:26.062990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-1cd2307b-5879-4913-a891-3c918b749759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:26.137999 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 472/935] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:10:26.339908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9a58a-a19d-4860-a0e8-c6971789b861 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:26.429191 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2307b-5879-4913-a891-3c918b749759 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:26.445265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9a58a-a19d-4860-a0e8-c6971789b861 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:26.485319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36102160-f9cb-47ad-ab8f-2b67028b38c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:26.486575 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36102160-f9cb-47ad-ab8f-2b67028b38c9 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:26.487069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36102160-f9cb-47ad-ab8f-2b67028b38c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:26.487478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36102160-f9cb-47ad-ab8f-2b67028b38c9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:26.501903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36102160-f9cb-47ad-ab8f-2b67028b38c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:10:26.507947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-36102160-f9cb-47ad-ab8f-2b67028b38c9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 12:10:26.507947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:26.507947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:26.507947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:26.507947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:26.507947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:26.507947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:26.507947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:26.507947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:26.507947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:26.507947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:26.507947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:26.507947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:10:26.507947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:10:26.507947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:10:26.507947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:10:26.509102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:10:26.509102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:10:26.509102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:10:26.509102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:10:26.509102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:26.509102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:26.509102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:26.509102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:26.509102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:26.509102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:26.509102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:26.509102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:26.509102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:26.509102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:26.509102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:26.509102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:26.509966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:26.509966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:26.509966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:26.509966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:26.509966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:10:26.509966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:10:26.509966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:10:26.509966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:10:26.509966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:10:26.509966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:10:26.509966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 12:10:26.509966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:26.512289 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 464/936] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Sep 28 12:10:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:26.518776 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36102160-f9cb-47ad-ab8f-2b67028b38c9 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:26.557274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2307b-5879-4913-a891-3c918b749759 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:26.587661 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:26.600277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ca69a-034e-4b9e-9acc-5a2f4361520f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:26.601197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ca69a-034e-4b9e-9acc-5a2f4361520f None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:26.601681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ca69a-034e-4b9e-9acc-5a2f4361520f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:26.602158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ca69a-034e-4b9e-9acc-5a2f4361520f None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:26.602978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ca69a-034e-4b9e-9acc-5a2f4361520f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:26.604573 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ca69a-034e-4b9e-9acc-5a2f4361520f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:26.605077 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:26.607825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: WARNING keystone.server.flask.common [None req-4c0ca69a-034e-4b9e-9acc-5a2f4361520f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 28 12:10:26.628079 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.sql.core [None req-4c0ca69a-034e-4b9e-9acc-5a2f4361520f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 28 12:10:26.628079 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 28 12:10:26.628079 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [parameters: {'id': '6ca7a9e63d314aadac6fc673a96abc00', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 28 12:10:26.628079 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81384) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 28 12:10:26.638604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-4c0ca69a-034e-4b9e-9acc-5a2f4361520f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6ca7a9e63d314aadac6fc673a96abc00.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6ca7a9e63d314aadac6fc673a96abc00. Sep 28 12:10:26.638604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:26.638604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 12:10:26.638604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 12:10:26.638604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 12:10:26.638604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 12:10:26.638604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 12:10:26.638604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 12:10:26.638604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 12:10:26.638604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application conn.query(q) Sep 28 12:10:26.638604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 12:10:26.638604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 12:10:26.638604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application result.read() Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 28 12:10:26.639482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 28 12:10:26.640990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 12:10:26.640990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application next(self.gen) Sep 28 12:10:26.640990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 28 12:10:26.640990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application yield resource Sep 28 12:10:26.640990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 12:10:26.640990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application next(self.gen) Sep 28 12:10:26.640990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 28 12:10:26.640990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self.session.rollback() Sep 28 12:10:26.640990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 28 12:10:26.640990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self.force_reraise() Sep 28 12:10:26.640990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 28 12:10:26.640990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise self.value Sep 28 12:10:26.640990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 28 12:10:26.640990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 28 12:10:26.640990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 28 12:10:26.640990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application session.commit() Sep 28 12:10:26.640990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 28 12:10:26.642109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 28 12:10:26.642109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 28 12:10:26.642109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._prepare_impl() Sep 28 12:10:26.642109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 28 12:10:26.642109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self.session.flush() Sep 28 12:10:26.642109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 28 12:10:26.642109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._flush(objects) Sep 28 12:10:26.642109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 28 12:10:26.642109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 28 12:10:26.642109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 28 12:10:26.642109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application compat.raise_( Sep 28 12:10:26.642109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 12:10:26.642109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception Sep 28 12:10:26.642109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 28 12:10:26.642109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application flush_context.execute() Sep 28 12:10:26.642109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 28 12:10:26.642109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rec.execute(self) Sep 28 12:10:26.643942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 28 12:10:26.643942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 28 12:10:26.643942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 28 12:10:26.643942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 28 12:10:26.643942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 28 12:10:26.643942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 28 12:10:26.643942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 28 12:10:26.643942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 28 12:10:26.643942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 28 12:10:26.643942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 28 12:10:26.643942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 28 12:10:26.643942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 28 12:10:26.643942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 28 12:10:26.643942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 28 12:10:26.643942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 28 12:10:26.645537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 28 12:10:26.645537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 12:10:26.645537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception Sep 28 12:10:26.645537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 12:10:26.645537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 12:10:26.645537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 12:10:26.645537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 12:10:26.645537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 12:10:26.645537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 12:10:26.645537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 12:10:26.645537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application conn.query(q) Sep 28 12:10:26.645537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 12:10:26.645537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 12:10:26.645537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 12:10:26.645537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application result.read() Sep 28 12:10:26.645537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 12:10:26.645537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 12:10:26.647530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 12:10:26.647530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 12:10:26.647530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 12:10:26.647530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 12:10:26.647530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 12:10:26.647530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 12:10:26.647530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 28 12:10:26.647530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 28 12:10:26.647530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [parameters: {'id': '6ca7a9e63d314aadac6fc673a96abc00', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 28 12:10:26.647530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 28 12:10:26.647530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:26.647530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:26.647530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:26.647530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:26.647530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 28 12:10:26.649564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 28 12:10:26.649564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 28 12:10:26.649564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 28 12:10:26.649564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 28 12:10:26.649564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:26.649564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:26.649564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:26.649564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:26.649564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:26.649564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:26.649564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:26.649564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:26.649564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:26.649564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:26.649564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:26.649564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:26.649564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:26.653159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:26.653159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 28 12:10:26.653159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 28 12:10:26.653159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:26.653159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:26.653159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 28 12:10:26.653159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 28 12:10:26.653159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6ca7a9e63d314aadac6fc673a96abc00. Sep 28 12:10:26.653159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:26.654403 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 473/937] 10.222.0.60 () {60 vars in 1140 bytes} [Wed Sep 28 12:10:26 2022] POST /identity/v3/projects => generated 262 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:26.666634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ca69a-034e-4b9e-9acc-5a2f4361520f None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:26.672637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36102160-f9cb-47ad-ab8f-2b67028b38c9 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:26.715091 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbb360-1265-4824-be36-cf12826cca4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:26.715681 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbb360-1265-4824-be36-cf12826cca4b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:26.715972 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbb360-1265-4824-be36-cf12826cca4b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:26.716294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbb360-1265-4824-be36-cf12826cca4b None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:26.716972 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbb360-1265-4824-be36-cf12826cca4b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:26.717313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbb360-1265-4824-be36-cf12826cca4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:26.719321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbb360-1265-4824-be36-cf12826cca4b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:26.728708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 465/938] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Sep 28 12:10:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:26.734783 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbb360-1265-4824-be36-cf12826cca4b None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:26.805479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ca69a-034e-4b9e-9acc-5a2f4361520f None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:26.850989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd15f7-e779-4e9c-8de6-ac31cec06793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:26.851772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd15f7-e779-4e9c-8de6-ac31cec06793 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:26.852058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd15f7-e779-4e9c-8de6-ac31cec06793 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:26.852335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd15f7-e779-4e9c-8de6-ac31cec06793 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:26.861563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd15f7-e779-4e9c-8de6-ac31cec06793 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:26.863234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd15f7-e779-4e9c-8de6-ac31cec06793 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:26.880696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-f5cd15f7-e779-4e9c-8de6-ac31cec06793 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:26.880696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:26.880696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:26.880696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:26.880696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:26.880696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:26.880696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:26.880696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:26.880696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:26.880696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:26.880696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:26.880696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:26.880696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:10:26.880696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:10:26.880696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:10:26.881693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:10:26.881693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:26.881693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:26.881693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:26.881693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:26.881693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:26.881693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:26.881693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:26.881693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:26.881693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:26.881693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:26.881693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:26.881693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:26.881693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:26.881693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:26.881693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:26.881693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:26.882802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:10:26.882802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:10:26.882802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:10:26.882802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:10:26.882802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:10:26.882802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:10:26.882802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:26.882802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:26.883625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbb360-1265-4824-be36-cf12826cca4b None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:26.883776 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 474/939] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:26.890399 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd15f7-e779-4e9c-8de6-ac31cec06793 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:26.908521 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:26.936201 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa669d4b-705b-4cfa-9848-79cf84fe25f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:26.937807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa669d4b-705b-4cfa-9848-79cf84fe25f4 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:26.938434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa669d4b-705b-4cfa-9848-79cf84fe25f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:26.938849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa669d4b-705b-4cfa-9848-79cf84fe25f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:26.952310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:26.955592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa669d4b-705b-4cfa-9848-79cf84fe25f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:26.960660 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa669d4b-705b-4cfa-9848-79cf84fe25f4 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:26.978314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-fa669d4b-705b-4cfa-9848-79cf84fe25f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:26.978314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:26.978314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:26.978314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:26.978314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:26.978314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:26.978314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:26.978314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:26.978314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:26.978314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:26.978314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:26.978314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:26.978314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:26.978314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:26.978314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:26.978314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:26.979372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:26.979372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:26.979372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:26.979372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:26.979372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:26.979372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:26.979372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:26.979372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:26.979372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:26.979372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:26.979372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:26.979372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:26.979372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:26.979372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:26.979372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:26.979372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:26.979372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:26.981017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:26.981017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:26.981017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:26.984094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 466/940] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:26.993195 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa669d4b-705b-4cfa-9848-79cf84fe25f4 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:27.021906 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd15f7-e779-4e9c-8de6-ac31cec06793 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:27.069094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9ba60-00cb-4496-bce3-5dc97d062985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:27.069771 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9ba60-00cb-4496-bce3-5dc97d062985 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:27.070221 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9ba60-00cb-4496-bce3-5dc97d062985 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:27.070592 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f9ba60-00cb-4496-bce3-5dc97d062985 None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:27.071262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f9ba60-00cb-4496-bce3-5dc97d062985 None admin] RBAC: Adding query filter params (name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:27.071654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f9ba60-00cb-4496-bce3-5dc97d062985 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:27.073352 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f9ba60-00cb-4496-bce3-5dc97d062985 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:27.105590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 475/941] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 12:10:26 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:27.118145 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa669d4b-705b-4cfa-9848-79cf84fe25f4 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:27.161803 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd724a-c0a9-4d9b-8672-06b92760c92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:27.163211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd724a-c0a9-4d9b-8672-06b92760c92f None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:27.163651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd724a-c0a9-4d9b-8672-06b92760c92f None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:27.164079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd724a-c0a9-4d9b-8672-06b92760c92f None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:27.164913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd724a-c0a9-4d9b-8672-06b92760c92f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:27.165657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd724a-c0a9-4d9b-8672-06b92760c92f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:27.167723 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd724a-c0a9-4d9b-8672-06b92760c92f None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:27.184510 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 467/942] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:27.189859 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9ba60-00cb-4496-bce3-5dc97d062985 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:27.312819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f9ba60-00cb-4496-bce3-5dc97d062985 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:27.315047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:27.353192 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efea729-0615-46bc-a43f-d0a525c8c356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:27.353822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efea729-0615-46bc-a43f-d0a525c8c356 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:27.354272 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efea729-0615-46bc-a43f-d0a525c8c356 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:27.354619 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efea729-0615-46bc-a43f-d0a525c8c356 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:27.363163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efea729-0615-46bc-a43f-d0a525c8c356 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:27.364648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efea729-0615-46bc-a43f-d0a525c8c356 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:27.377662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-2efea729-0615-46bc-a43f-d0a525c8c356 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 12:10:27.377662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:27.377662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:27.377662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:27.377662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:27.377662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:27.377662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:27.377662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:27.377662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:27.377662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:27.377662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:27.377662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:27.377662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:10:27.377662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:10:27.377662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:10:27.377662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:27.378637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:27.379565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:27.379565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:27.379565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:10:27.379565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:10:27.379565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:10:27.379565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:10:27.379565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:10:27.379565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:10:27.379565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 12:10:27.379565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:27.379565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 476/943] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 12:10:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:27.386297 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd724a-c0a9-4d9b-8672-06b92760c92f None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:27.417629 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:27.505200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd724a-c0a9-4d9b-8672-06b92760c92f None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:27.547514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5019f-7a8c-416e-9b02-c1762e236d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:27.548127 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5019f-7a8c-416e-9b02-c1762e236d9d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:27.548402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5019f-7a8c-416e-9b02-c1762e236d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:27.548667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5019f-7a8c-416e-9b02-c1762e236d9d None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:27.549594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5019f-7a8c-416e-9b02-c1762e236d9d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:27.549898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5019f-7a8c-416e-9b02-c1762e236d9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:27.551306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5019f-7a8c-416e-9b02-c1762e236d9d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:27.569513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 468/944] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Sep 28 12:10:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:27.575181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efea729-0615-46bc-a43f-d0a525c8c356 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:27.685991 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efea729-0615-46bc-a43f-d0a525c8c356 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:27.727921 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c92560-06f5-4956-812d-2f66f77168d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:27.728554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c92560-06f5-4956-812d-2f66f77168d5 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:27.728830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c92560-06f5-4956-812d-2f66f77168d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:27.729247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c92560-06f5-4956-812d-2f66f77168d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:27.740774 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c92560-06f5-4956-812d-2f66f77168d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:27.742245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c92560-06f5-4956-812d-2f66f77168d5 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:27.757433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-b5c92560-06f5-4956-812d-2f66f77168d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:27.757433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:27.757433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:27.757433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:27.757433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:27.757433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:27.757433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:27.757433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:27.757433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:27.757433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:27.757433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:27.757433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:27.757433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:27.757433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:27.757433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:27.757433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:27.758410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:27.758410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:27.758410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:27.758410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:27.758410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:27.758410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:27.758410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:27.758410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:27.758410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:27.758410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:27.758410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:27.758410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:27.758410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:27.758410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:27.758410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:27.758410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:27.758410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:27.759302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:27.759302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:27.759302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:27.759302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 477/945] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:27.766586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5019f-7a8c-416e-9b02-c1762e236d9d None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:27.859508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:27.879461 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5019f-7a8c-416e-9b02-c1762e236d9d None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:27.918000 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a376ffb-a6f5-48a9-b74d-893b8cdc61dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:27.918620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a376ffb-a6f5-48a9-b74d-893b8cdc61dd None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:27.918904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a376ffb-a6f5-48a9-b74d-893b8cdc61dd None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:27.919179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a376ffb-a6f5-48a9-b74d-893b8cdc61dd None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:27.919824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a376ffb-a6f5-48a9-b74d-893b8cdc61dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:27.920129 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a376ffb-a6f5-48a9-b74d-893b8cdc61dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:27.921590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a376ffb-a6f5-48a9-b74d-893b8cdc61dd None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:27.933759 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 469/946] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:27.940142 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c92560-06f5-4956-812d-2f66f77168d5 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:27.957715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:27.986113 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fc361-55f3-4c67-82f9-eeca558487e5 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:27.986457 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fc361-55f3-4c67-82f9-eeca558487e5 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:27.986665 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8fc361-55f3-4c67-82f9-eeca558487e5 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:27.988345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 470/947] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:10:27.994638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de72057-bcb6-4188-89dd-3d166a2931f3 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:27.994980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de72057-bcb6-4188-89dd-3d166a2931f3 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:27.995258 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de72057-bcb6-4188-89dd-3d166a2931f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:28.013336 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-5de72057-bcb6-4188-89dd-3d166a2931f3 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:28.014564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5de72057-bcb6-4188-89dd-3d166a2931f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:28.066549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c92560-06f5-4956-812d-2f66f77168d5 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:28.093250 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 471/948] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:10:28.104680 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fabbf7-64a0-492c-a05a-be4425b8a723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:28.105449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fabbf7-64a0-492c-a05a-be4425b8a723 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:28.105823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fabbf7-64a0-492c-a05a-be4425b8a723 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:28.106167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fabbf7-64a0-492c-a05a-be4425b8a723 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:28.115137 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fabbf7-64a0-492c-a05a-be4425b8a723 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:28.116694 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fabbf7-64a0-492c-a05a-be4425b8a723 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:28.131052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-13fabbf7-64a0-492c-a05a-be4425b8a723 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:28.131052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:28.131052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:28.131052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:28.131052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:28.131052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:28.131052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:28.131052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:28.131052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:28.131052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:28.131052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:28.131052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:28.131052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:10:28.131052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:10:28.131052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:10:28.132003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:10:28.132003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:28.132003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:28.132003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:28.132003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:28.132003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:28.132003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:28.132003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:28.132003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:28.132003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:28.132003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:28.132003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:28.132003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:28.132003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:28.132003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:28.132003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:28.132003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:28.133233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:10:28.133233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:10:28.133233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:10:28.133233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:10:28.133233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:10:28.133233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:10:28.133233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:28.133233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:28.133233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 478/949] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:28.138588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de72057-bcb6-4188-89dd-3d166a2931f3 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:28.226065 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:28.251486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e022bf-023b-422a-8176-399c807f7eb6 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:28.251831 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e022bf-023b-422a-8176-399c807f7eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:28.252038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e022bf-023b-422a-8176-399c807f7eb6 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:28.253680 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 479/950] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:10:28.254366 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de72057-bcb6-4188-89dd-3d166a2931f3 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:28.258668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d94d5c-01c4-40a2-ae24-41b03abff786 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:28.258952 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d94d5c-01c4-40a2-ae24-41b03abff786 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:28.259230 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d94d5c-01c4-40a2-ae24-41b03abff786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:28.281722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-38d94d5c-01c4-40a2-ae24-41b03abff786 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:28.283308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-38d94d5c-01c4-40a2-ae24-41b03abff786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:28.292343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bae2d7-3dec-4ec9-b8dc-90ff612bb699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:28.292958 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bae2d7-3dec-4ec9-b8dc-90ff612bb699 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:28.293266 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bae2d7-3dec-4ec9-b8dc-90ff612bb699 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:28.293538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bae2d7-3dec-4ec9-b8dc-90ff612bb699 None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:28.294227 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bae2d7-3dec-4ec9-b8dc-90ff612bb699 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:28.294532 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bae2d7-3dec-4ec9-b8dc-90ff612bb699 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:28.296539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bae2d7-3dec-4ec9-b8dc-90ff612bb699 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:28.312226 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 472/951] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:10:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:28.325153 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bae2d7-3dec-4ec9-b8dc-90ff612bb699 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:28.365811 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 480/952] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:10:28.385745 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d94d5c-01c4-40a2-ae24-41b03abff786 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:28.390512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:28.445263 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bae2d7-3dec-4ec9-b8dc-90ff612bb699 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:28.480188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240d8a3-8540-4ded-b056-699cfe58e799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:28.480908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240d8a3-8540-4ded-b056-699cfe58e799 None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:28.481231 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240d8a3-8540-4ded-b056-699cfe58e799 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:28.481512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240d8a3-8540-4ded-b056-699cfe58e799 None admin] PATH_INFO: `/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/bc69cf267bea41bf890bc7551d84ad6f/roles/8af05c40a1604332a08a2c6ba7ad6de7` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:28.483824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240d8a3-8540-4ded-b056-699cfe58e799 None admin] RBAC: Authorizing `identity:create_grant(project_id=6335d0db1f4844c39c255909189e0ce9, user_id=bc69cf267bea41bf890bc7551d84ad6f, role_id=8af05c40a1604332a08a2c6ba7ad6de7)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:28.485276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240d8a3-8540-4ded-b056-699cfe58e799 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:28.487711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:28.505657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 473/953] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:10:28 2022] PUT /identity/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/bc69cf267bea41bf890bc7551d84ad6f/roles/8af05c40a1604332a08a2c6ba7ad6de7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:10:28.517485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d94d5c-01c4-40a2-ae24-41b03abff786 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:28.557699 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f654987-fb6b-4b63-abf8-ece0c8ac410e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:28.558457 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f654987-fb6b-4b63-abf8-ece0c8ac410e None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:28.558783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f654987-fb6b-4b63-abf8-ece0c8ac410e None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:28.559112 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f654987-fb6b-4b63-abf8-ece0c8ac410e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:28.559699 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f654987-fb6b-4b63-abf8-ece0c8ac410e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:28.561078 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f654987-fb6b-4b63-abf8-ece0c8ac410e None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:28.572039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-5f654987-fb6b-4b63-abf8-ece0c8ac410e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 12:10:28.572039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:28.572039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:28.572039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:28.572039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:28.572039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:28.572039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:28.572039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:28.572039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 12:10:28.572039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:28.572039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:28.572039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:28.572039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:28.572039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:28.572039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:28.572039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:28.572039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:28.574686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:28.574686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:28.574686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:28.574686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:28.574686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:28.574686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:28.574686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:28.574686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:28.574686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:28.574686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:28.574686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:28.574686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:28.574686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:28.574686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:28.574686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:28.574686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:28.577838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:28.577838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:28.577838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:28.577838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:28.577838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:28.577838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:28.577838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:28.577838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:28.577838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:28.577838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:28.577838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:28.577838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:28.577838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 12:10:28.577838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:28.579736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 481/954] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:28.585158 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240d8a3-8540-4ded-b056-699cfe58e799 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:28.654268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f654987-fb6b-4b63-abf8-ece0c8ac410e None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:28.719063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240d8a3-8540-4ded-b056-699cfe58e799 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:28.745491 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:28.762396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8b32d-5808-4b7e-b08c-8788c52ce80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:28.763132 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8b32d-5808-4b7e-b08c-8788c52ce80f None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:28.763414 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8b32d-5808-4b7e-b08c-8788c52ce80f None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:28.763683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8b32d-5808-4b7e-b08c-8788c52ce80f None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:28.764465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e8b32d-5808-4b7e-b08c-8788c52ce80f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:28.765386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e8b32d-5808-4b7e-b08c-8788c52ce80f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:28.767425 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e8b32d-5808-4b7e-b08c-8788c52ce80f None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:28.777242 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 474/955] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 12:10:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:28.779215 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8b32d-5808-4b7e-b08c-8788c52ce80f None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:28.788501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f654987-fb6b-4b63-abf8-ece0c8ac410e None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:28.832936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deeb9ae-c585-4269-bdf6-08f58db669f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:28.834021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deeb9ae-c585-4269-bdf6-08f58db669f8 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:28.834313 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deeb9ae-c585-4269-bdf6-08f58db669f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:28.834592 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deeb9ae-c585-4269-bdf6-08f58db669f8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:28.835455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deeb9ae-c585-4269-bdf6-08f58db669f8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:28.837333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deeb9ae-c585-4269-bdf6-08f58db669f8 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:28.841675 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:28.850527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-8deeb9ae-c585-4269-bdf6-08f58db669f8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 12:10:28.850527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:28.850527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:28.850527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:28.850527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:28.850527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:28.850527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:28.850527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:28.850527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 12:10:28.850527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:28.850527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:28.850527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:28.850527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:28.850527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:28.850527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:28.850527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:28.850527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:28.851602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:28.851602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:28.851602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:28.851602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:28.851602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:28.851602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:28.851602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:28.851602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:28.851602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:28.851602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:28.851602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:28.851602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:28.851602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:28.851602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:28.851602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:28.851602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:28.852487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:28.852487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:28.852487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:28.852487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:28.852487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:28.852487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:28.852487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:28.852487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:28.852487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:28.852487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:28.852487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:28.852487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:28.852487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 12:10:28.852487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:28.852487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 482/956] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 12:10:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:28.866117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deeb9ae-c585-4269-bdf6-08f58db669f8 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:28.892285 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8b32d-5808-4b7e-b08c-8788c52ce80f None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:28.942136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5e9d5-b17a-4bc0-ad6b-bbf7b3b2e0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:28.942757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5e9d5-b17a-4bc0-ad6b-bbf7b3b2e0b7 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:28.943040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5e9d5-b17a-4bc0-ad6b-bbf7b3b2e0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:28.943312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5e9d5-b17a-4bc0-ad6b-bbf7b3b2e0b7 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:28.944523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5e9d5-b17a-4bc0-ad6b-bbf7b3b2e0b7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:28.944836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5e9d5-b17a-4bc0-ad6b-bbf7b3b2e0b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:28.946560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5e9d5-b17a-4bc0-ad6b-bbf7b3b2e0b7 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:28.955717 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 475/957] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 12:10:28 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:28.984116 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deeb9ae-c585-4269-bdf6-08f58db669f8 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:29.019513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:29.025598 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20eea3c-3ccc-4508-bde6-2abb3b8c85fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:29.026234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20eea3c-3ccc-4508-bde6-2abb3b8c85fa None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:29.026961 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20eea3c-3ccc-4508-bde6-2abb3b8c85fa None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:29.027441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20eea3c-3ccc-4508-bde6-2abb3b8c85fa None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:29.028433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20eea3c-3ccc-4508-bde6-2abb3b8c85fa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:29.028738 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20eea3c-3ccc-4508-bde6-2abb3b8c85fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:29.030251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20eea3c-3ccc-4508-bde6-2abb3b8c85fa None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:29.036353 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 483/958] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 12:10:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:29.042260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5e9d5-b17a-4bc0-ad6b-bbf7b3b2e0b7 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:29.160509 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5e9d5-b17a-4bc0-ad6b-bbf7b3b2e0b7 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:29.185336 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:29.199386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a6a91-cafc-4627-a1a6-2b35a494de7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:29.199927 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a6a91-cafc-4627-a1a6-2b35a494de7c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:29.200206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a6a91-cafc-4627-a1a6-2b35a494de7c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:29.200479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a6a91-cafc-4627-a1a6-2b35a494de7c None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:29.201109 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a6a91-cafc-4627-a1a6-2b35a494de7c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:29.201440 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a6a91-cafc-4627-a1a6-2b35a494de7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:29.202945 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a6a91-cafc-4627-a1a6-2b35a494de7c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:29.208355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 476/959] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 12:10:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:29.213730 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20eea3c-3ccc-4508-bde6-2abb3b8c85fa None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:29.318724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20eea3c-3ccc-4508-bde6-2abb3b8c85fa None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:29.355858 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff9b23-04c5-403f-89d7-6553bb53f312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:29.356477 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff9b23-04c5-403f-89d7-6553bb53f312 None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:29.356756 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff9b23-04c5-403f-89d7-6553bb53f312 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:29.357027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff9b23-04c5-403f-89d7-6553bb53f312 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:29.357726 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff9b23-04c5-403f-89d7-6553bb53f312 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:29.359674 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff9b23-04c5-403f-89d7-6553bb53f312 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:29.383305 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 484/960] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:10:29 2022] POST /identity/v3/endpoints => generated 349 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:10:29.625297 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d001d3-a6f1-402a-86db-fbbd62beffb4 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:29.625791 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d001d3-a6f1-402a-86db-fbbd62beffb4 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:29.625791 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d001d3-a6f1-402a-86db-fbbd62beffb4 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:29.627342 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 477/961] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:29.632878 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3615234-2b02-4702-8f8c-daa708731e34 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:29.633346 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3615234-2b02-4702-8f8c-daa708731e34 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:29.633528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3615234-2b02-4702-8f8c-daa708731e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:29.652128 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-d3615234-2b02-4702-8f8c-daa708731e34 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:29.653328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d3615234-2b02-4702-8f8c-daa708731e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:29.701008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:29.728776 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 485/962] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:30.014819 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d001d3-a6f1-402a-86db-fbbd62beffb4 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:30.119112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d001d3-a6f1-402a-86db-fbbd62beffb4 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:30.154420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bf597-360a-4e0f-adf8-134051a233e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:30.155026 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bf597-360a-4e0f-adf8-134051a233e6 None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:30.155885 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bf597-360a-4e0f-adf8-134051a233e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:30.156153 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bf597-360a-4e0f-adf8-134051a233e6 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:30.156790 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bf597-360a-4e0f-adf8-134051a233e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:30.158240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bf597-360a-4e0f-adf8-134051a233e6 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:30.160874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: WARNING keystone.server.flask.common [None req-b72bf597-360a-4e0f-adf8-134051a233e6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 28 12:10:30.176811 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.sql.core [None req-b72bf597-360a-4e0f-adf8-134051a233e6 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 28 12:10:30.176811 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 28 12:10:30.176811 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [parameters: {'user_id': 'd95ede4ba4f74f4aa916f82b25675a15', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 28 12:10:30.176811 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81385) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 28 12:10:30.194524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-b72bf597-360a-4e0f-adf8-134051a233e6 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 28 12:10:30.194524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:30.194524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 12:10:30.194524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 12:10:30.194524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 12:10:30.194524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 12:10:30.194524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 12:10:30.194524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 12:10:30.194524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 12:10:30.194524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application conn.query(q) Sep 28 12:10:30.194524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 12:10:30.194524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 12:10:30.194524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 12:10:30.194524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application result.read() Sep 28 12:10:30.194524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 12:10:30.195479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application next(self.gen) Sep 28 12:10:30.196448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 28 12:10:30.196448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application yield resource Sep 28 12:10:30.196448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 12:10:30.196448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application next(self.gen) Sep 28 12:10:30.196448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 28 12:10:30.196448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self.session.rollback() Sep 28 12:10:30.196448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 28 12:10:30.196448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self.force_reraise() Sep 28 12:10:30.196448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 28 12:10:30.196448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise self.value Sep 28 12:10:30.196448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 28 12:10:30.196448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 28 12:10:30.196448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 28 12:10:30.196448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application session.commit() Sep 28 12:10:30.196448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 28 12:10:30.196448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 28 12:10:30.196448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 28 12:10:30.197404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._prepare_impl() Sep 28 12:10:30.197404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 28 12:10:30.197404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self.session.flush() Sep 28 12:10:30.197404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 28 12:10:30.197404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._flush(objects) Sep 28 12:10:30.197404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 28 12:10:30.197404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 28 12:10:30.197404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 28 12:10:30.197404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application compat.raise_( Sep 28 12:10:30.197404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 12:10:30.197404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception Sep 28 12:10:30.197404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 28 12:10:30.197404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application flush_context.execute() Sep 28 12:10:30.197404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 28 12:10:30.197404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rec.execute(self) Sep 28 12:10:30.197404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 28 12:10:30.197404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 28 12:10:30.198329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 28 12:10:30.198329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 28 12:10:30.198329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 28 12:10:30.198329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 28 12:10:30.198329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 28 12:10:30.198329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 28 12:10:30.198329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 28 12:10:30.198329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 28 12:10:30.198329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 28 12:10:30.198329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 28 12:10:30.198329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 28 12:10:30.198329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 28 12:10:30.198329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 28 12:10:30.198329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 28 12:10:30.198329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 12:10:30.198329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception Sep 28 12:10:30.200107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 12:10:30.200107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 12:10:30.200107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 12:10:30.200107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 12:10:30.200107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 12:10:30.200107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 12:10:30.200107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 12:10:30.200107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application conn.query(q) Sep 28 12:10:30.200107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 12:10:30.200107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 12:10:30.200107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 12:10:30.200107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application result.read() Sep 28 12:10:30.200107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 12:10:30.200107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 12:10:30.200107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 12:10:30.200107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 12:10:30.202248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 12:10:30.202248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 12:10:30.202248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 12:10:30.202248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 12:10:30.202248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 28 12:10:30.202248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 28 12:10:30.202248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd95ede4ba4f74f4aa916f82b25675a15', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 28 12:10:30.202248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 28 12:10:30.202248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:30.202248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:30.202248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:30.202248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:30.202248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:30.202248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:30.202248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:30.204341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:30.204341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:30.204341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:30.204341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:30.204341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:30.204341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:30.204341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:30.204341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 28 12:10:30.204341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 28 12:10:30.204341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:30.204341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:30.204341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:10:30.204341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:30.204341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:10:30.204341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:30.204341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 28 12:10:30.204341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 28 12:10:30.206538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 28 12:10:30.206538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 28 12:10:30.206538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 28 12:10:30.206538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 28 12:10:30.206538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 28 12:10:30.206538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:30.209152 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 478/963] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Sep 28 12:10:30 2022] POST /identity/v3/users => generated 160 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:30.218975 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3615234-2b02-4702-8f8c-daa708731e34 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:30.329070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3615234-2b02-4702-8f8c-daa708731e34 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:30.365273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b176b9ad-6cdc-469b-b962-014badf951a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:30.365908 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176b9ad-6cdc-469b-b962-014badf951a2 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:30.366180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176b9ad-6cdc-469b-b962-014badf951a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:30.366446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b176b9ad-6cdc-469b-b962-014badf951a2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:30.373610 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176b9ad-6cdc-469b-b962-014badf951a2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:30.375149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b176b9ad-6cdc-469b-b962-014badf951a2 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:30.386724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-b176b9ad-6cdc-469b-b962-014badf951a2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 12:10:30.386724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:30.386724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:30.386724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:30.386724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:30.386724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:30.386724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:30.386724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:30.386724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:30.386724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:30.386724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:30.386724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:30.386724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:10:30.386724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:10:30.386724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:10:30.386724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:30.388916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:30.391720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:30.391720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:30.391720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:10:30.391720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:10:30.391720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:10:30.391720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:10:30.391720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:10:30.391720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:10:30.391720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 12:10:30.391720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:30.391720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 486/964] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:10:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:30.397174 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bf597-360a-4e0f-adf8-134051a233e6 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:30.449750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:30.507572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bf597-360a-4e0f-adf8-134051a233e6 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:30.541543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2b9ce-26b2-4293-9e27-51f85a9a9047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:30.542221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2b9ce-26b2-4293-9e27-51f85a9a9047 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:30.542505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2b9ce-26b2-4293-9e27-51f85a9a9047 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:30.542778 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2b9ce-26b2-4293-9e27-51f85a9a9047 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:30.543469 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2b9ce-26b2-4293-9e27-51f85a9a9047 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:30.543776 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2b9ce-26b2-4293-9e27-51f85a9a9047 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:30.545468 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2b9ce-26b2-4293-9e27-51f85a9a9047 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:30.584987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 479/965] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Sep 28 12:10:30 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:30.838795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754c2d8-4974-4fee-ad9a-b998eadf233f None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:30.839281 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754c2d8-4974-4fee-ad9a-b998eadf233f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:30.839281 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754c2d8-4974-4fee-ad9a-b998eadf233f None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:30.840733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 487/966] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:30.846974 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e78d0-1057-42d2-bd92-f73d129b758c None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:30.847401 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e78d0-1057-42d2-bd92-f73d129b758c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:30.847580 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e78d0-1057-42d2-bd92-f73d129b758c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:30.870836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-9d6e78d0-1057-42d2-bd92-f73d129b758c None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:30.872369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-9d6e78d0-1057-42d2-bd92-f73d129b758c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:30.942337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 480/967] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:31.209700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754c2d8-4974-4fee-ad9a-b998eadf233f None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:31.250286 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:31.286408 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99c6a5-f1c4-401e-9f28-d6b3a4d3c221 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:31.286707 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99c6a5-f1c4-401e-9f28-d6b3a4d3c221 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:31.286972 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99c6a5-f1c4-401e-9f28-d6b3a4d3c221 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:31.288303 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 481/968] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:31.293414 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39c97a-f0e2-4227-b3de-9d22051fd3ae None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:31.293696 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39c97a-f0e2-4227-b3de-9d22051fd3ae None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:31.293968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39c97a-f0e2-4227-b3de-9d22051fd3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:31.313472 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-3f39c97a-f0e2-4227-b3de-9d22051fd3ae None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:31.314496 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754c2d8-4974-4fee-ad9a-b998eadf233f None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:31.314760 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-3f39c97a-f0e2-4227-b3de-9d22051fd3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:31.346197 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:31.350263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85d0d8-4168-4689-a8e4-7fd0bdf27588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:31.350895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85d0d8-4168-4689-a8e4-7fd0bdf27588 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:31.351279 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85d0d8-4168-4689-a8e4-7fd0bdf27588 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:31.351463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85d0d8-4168-4689-a8e4-7fd0bdf27588 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:31.363030 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85d0d8-4168-4689-a8e4-7fd0bdf27588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:10:31.367830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-ff85d0d8-4168-4689-a8e4-7fd0bdf27588 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 12:10:31.367830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:31.367830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:31.367830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:31.367830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:31.367830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:31.367830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:31.367830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:31.367830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:31.367830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:31.367830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:31.367830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:31.367830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:10:31.367830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:10:31.367830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:10:31.367830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:10:31.368857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:10:31.368857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:10:31.368857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:10:31.368857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:10:31.368857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:31.368857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:31.368857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:31.368857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:31.368857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:31.368857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:31.368857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:31.368857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:31.368857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:31.368857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:31.368857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:31.368857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:31.369743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:31.369743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:31.369743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:31.369743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:31.369743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:10:31.369743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:10:31.369743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:10:31.369743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:10:31.369743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:10:31.369743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:10:31.369743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 12:10:31.369743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:31.369743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 488/969] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Sep 28 12:10:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:31.375624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85d0d8-4168-4689-a8e4-7fd0bdf27588 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:31.393173 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 482/970] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:10:31.473867 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:31.478257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85d0d8-4168-4689-a8e4-7fd0bdf27588 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:31.512555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbc28c-1174-43d3-b396-6ec460ee2cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:31.513181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbc28c-1174-43d3-b396-6ec460ee2cfa None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:31.513466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbc28c-1174-43d3-b396-6ec460ee2cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:31.513736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbc28c-1174-43d3-b396-6ec460ee2cfa None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:31.514386 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbc28c-1174-43d3-b396-6ec460ee2cfa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:31.515246 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbc28c-1174-43d3-b396-6ec460ee2cfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:31.516607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fbc28c-1174-43d3-b396-6ec460ee2cfa None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:31.524618 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 489/971] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Sep 28 12:10:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:31.530367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39c97a-f0e2-4227-b3de-9d22051fd3ae None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:31.631081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39c97a-f0e2-4227-b3de-9d22051fd3ae None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:31.668722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbc28c-1174-43d3-b396-6ec460ee2cfa None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:31.675596 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f111dd-86bb-470a-8537-1fb07b19595b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:31.676237 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f111dd-86bb-470a-8537-1fb07b19595b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:31.676515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f111dd-86bb-470a-8537-1fb07b19595b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:31.676789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f111dd-86bb-470a-8537-1fb07b19595b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:31.687270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f111dd-86bb-470a-8537-1fb07b19595b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:31.688696 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f111dd-86bb-470a-8537-1fb07b19595b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:31.703035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-85f111dd-86bb-470a-8537-1fb07b19595b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:31.703035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:31.703035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:31.703035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:31.703035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:31.703035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:31.703035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:31.703035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:31.703035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:31.703035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:31.703035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:31.703035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:31.703035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:31.703035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:31.703035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:31.703035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:31.703992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:31.703992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:31.703992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:31.703992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:31.703992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:31.703992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:31.703992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:31.703992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:31.703992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:31.703992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:31.703992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:31.703992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:31.703992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:31.703992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:31.703992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:31.703992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:31.703992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:31.704874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:31.704874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:31.704874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:31.708994 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 483/972] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:31.709840 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f111dd-86bb-470a-8537-1fb07b19595b None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:31.729627 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:31.777354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbc28c-1174-43d3-b396-6ec460ee2cfa None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:31.811208 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c15bd-30c1-49c9-9290-0bc857953664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:31.811594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f111dd-86bb-470a-8537-1fb07b19595b None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:31.811903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c15bd-30c1-49c9-9290-0bc857953664 None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:31.812256 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c15bd-30c1-49c9-9290-0bc857953664 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:31.812527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c15bd-30c1-49c9-9290-0bc857953664 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:31.813184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c15bd-30c1-49c9-9290-0bc857953664 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:31.814530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c15bd-30c1-49c9-9290-0bc857953664 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:31.829433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 490/973] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 12:10:31 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:10:31.830524 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c819d-f35e-4856-8397-ac27818943e7 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:31.830808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c819d-f35e-4856-8397-ac27818943e7 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:31.831080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c819d-f35e-4856-8397-ac27818943e7 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:31.833190 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 491/974] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:31.838291 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281e29c-ab3d-47f4-a57b-3d3b51f81864 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:31.838612 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281e29c-ab3d-47f4-a57b-3d3b51f81864 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:31.838885 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281e29c-ab3d-47f4-a57b-3d3b51f81864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:31.856944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f81f3e-56ac-4d18-818d-c46b1b147c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:31.857675 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f81f3e-56ac-4d18-818d-c46b1b147c5e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:31.857865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f81f3e-56ac-4d18-818d-c46b1b147c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:31.858134 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f81f3e-56ac-4d18-818d-c46b1b147c5e None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:31.858854 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f81f3e-56ac-4d18-818d-c46b1b147c5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:31.859157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f81f3e-56ac-4d18-818d-c46b1b147c5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:31.860573 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f81f3e-56ac-4d18-818d-c46b1b147c5e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:31.865334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-a281e29c-ab3d-47f4-a57b-3d3b51f81864 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:31.866588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a281e29c-ab3d-47f4-a57b-3d3b51f81864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:31.874568 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 484/975] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:31.880858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f81f3e-56ac-4d18-818d-c46b1b147c5e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:31.887642 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:31.961169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 492/976] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:31.973212 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:32.002204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f81f3e-56ac-4d18-818d-c46b1b147c5e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:32.039678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df33ca-7ba6-4919-aea9-864e449237b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:32.040311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df33ca-7ba6-4919-aea9-864e449237b9 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:32.040594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df33ca-7ba6-4919-aea9-864e449237b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:32.040867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df33ca-7ba6-4919-aea9-864e449237b9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:32.049105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df33ca-7ba6-4919-aea9-864e449237b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:32.050577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df33ca-7ba6-4919-aea9-864e449237b9 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:32.064085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-27df33ca-7ba6-4919-aea9-864e449237b9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 12:10:32.064085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:32.064085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:32.064085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:32.064085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:32.064085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:32.064085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:32.064085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:32.064085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:32.064085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:32.064085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:32.064085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:32.064085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:10:32.064085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:10:32.064085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:10:32.064085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:32.065074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:32.066105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:32.066105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:32.066105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:10:32.066105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:10:32.066105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:10:32.066105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:10:32.066105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:10:32.066105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:10:32.066105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 12:10:32.066105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:32.069225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 485/977] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 12:10:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:32.071189 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281e29c-ab3d-47f4-a57b-3d3b51f81864 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:32.181933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281e29c-ab3d-47f4-a57b-3d3b51f81864 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:32.217304 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce309d7-27ac-46a6-a30e-1ef71472a080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:32.217941 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce309d7-27ac-46a6-a30e-1ef71472a080 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:32.218214 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce309d7-27ac-46a6-a30e-1ef71472a080 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:32.218481 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce309d7-27ac-46a6-a30e-1ef71472a080 None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:32.219193 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce309d7-27ac-46a6-a30e-1ef71472a080 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:32.219496 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce309d7-27ac-46a6-a30e-1ef71472a080 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:32.220898 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce309d7-27ac-46a6-a30e-1ef71472a080 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:32.243034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 493/978] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Sep 28 12:10:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:32.244611 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df33ca-7ba6-4919-aea9-864e449237b9 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:32.245897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce309d7-27ac-46a6-a30e-1ef71472a080 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:32.257206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:32.352801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce309d7-27ac-46a6-a30e-1ef71472a080 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:32.357447 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df33ca-7ba6-4919-aea9-864e449237b9 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:32.390450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07677b08-2be7-497a-a809-399269a54ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:32.391175 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07677b08-2be7-497a-a809-399269a54ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:32.391459 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07677b08-2be7-497a-a809-399269a54ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:32.391733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07677b08-2be7-497a-a809-399269a54ef1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:32.392304 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07677b08-2be7-497a-a809-399269a54ef1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:32.393702 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07677b08-2be7-497a-a809-399269a54ef1 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:32.394116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172926b-3908-40db-bead-0dd28ed3c063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:32.394718 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172926b-3908-40db-bead-0dd28ed3c063 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:32.394995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172926b-3908-40db-bead-0dd28ed3c063 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:32.395261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172926b-3908-40db-bead-0dd28ed3c063 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:32.397962 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:32.404627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-07677b08-2be7-497a-a809-399269a54ef1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 12:10:32.404627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:32.404627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:32.404627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:32.404627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:32.404627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:32.404627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:32.404627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:32.404627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 12:10:32.404627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:32.404627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:32.404627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:32.404627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:32.404627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:32.404627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:32.404627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:32.404627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:32.409571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:32.409571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:32.409571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:32.409571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:32.409571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:32.409571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:32.409571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:32.409571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:32.409571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:32.409571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:32.409571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:32.409571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:32.409571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:32.409571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:32.409571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:32.409571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:32.410431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:32.410431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:32.410431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:32.410431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:32.410431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:32.410431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:32.410431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:32.410431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:32.410431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:32.410431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:32.410431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:32.410431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:32.410431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 12:10:32.410431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:32.410431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 494/979] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 12:10:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:32.412323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07677b08-2be7-497a-a809-399269a54ef1 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:32.468671 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6172926b-3908-40db-bead-0dd28ed3c063 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:32.470163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6172926b-3908-40db-bead-0dd28ed3c063 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:32.486429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:32.519826 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07677b08-2be7-497a-a809-399269a54ef1 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:32.556742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1538b4f-b5aa-40b4-a2e4-6ad7d11be124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:32.557674 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1538b4f-b5aa-40b4-a2e4-6ad7d11be124 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:32.557958 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1538b4f-b5aa-40b4-a2e4-6ad7d11be124 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:32.558232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1538b4f-b5aa-40b4-a2e4-6ad7d11be124 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:32.558856 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1538b4f-b5aa-40b4-a2e4-6ad7d11be124 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:32.559157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1538b4f-b5aa-40b4-a2e4-6ad7d11be124 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:32.560940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1538b4f-b5aa-40b4-a2e4-6ad7d11be124 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:32.565846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 495/980] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 12:10:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:32.571480 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1538b4f-b5aa-40b4-a2e4-6ad7d11be124 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:32.578551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 486/981] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:10:32 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:10:32.581002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172926b-3908-40db-bead-0dd28ed3c063 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:32.616706 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:32.683930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1538b4f-b5aa-40b4-a2e4-6ad7d11be124 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:32.687752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172926b-3908-40db-bead-0dd28ed3c063 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:32.719237 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bdabf-feef-432c-b6cb-d11b796ca621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:32.720112 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bdabf-feef-432c-b6cb-d11b796ca621 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:32.720563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bdabf-feef-432c-b6cb-d11b796ca621 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:32.721005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bdabf-feef-432c-b6cb-d11b796ca621 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:32.721837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342bdabf-feef-432c-b6cb-d11b796ca621 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:32.722312 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342bdabf-feef-432c-b6cb-d11b796ca621 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:32.722647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea59704-7ded-413c-9779-f7821ade53fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:32.723280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea59704-7ded-413c-9779-f7821ade53fe None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:32.723558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea59704-7ded-413c-9779-f7821ade53fe None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:32.723831 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea59704-7ded-413c-9779-f7821ade53fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:32.724604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342bdabf-feef-432c-b6cb-d11b796ca621 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:32.727124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:32.731107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 496/982] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 12:10:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:32.738514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bdabf-feef-432c-b6cb-d11b796ca621 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:32.741861 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea59704-7ded-413c-9779-f7821ade53fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:32.743340 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea59704-7ded-413c-9779-f7821ade53fe None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:32.759975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-7ea59704-7ded-413c-9779-f7821ade53fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:32.759975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:32.759975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:32.759975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:32.759975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:32.759975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:32.759975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:32.759975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:32.759975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:32.759975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:32.759975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:32.759975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:32.759975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:32.759975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:32.759975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:32.759975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:32.760983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:32.760983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:32.760983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:32.760983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:32.760983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:32.760983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:32.760983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:32.760983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:32.760983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:32.760983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:32.760983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:32.760983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:32.760983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:32.760983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:32.760983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:32.760983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:32.760983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:32.761909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:32.761909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:32.761909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:32.761909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 487/983] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:32.766812 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea59704-7ded-413c-9779-f7821ade53fe None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:32.851893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bdabf-feef-432c-b6cb-d11b796ca621 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:32.855153 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:32.872728 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea59704-7ded-413c-9779-f7821ade53fe None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:32.891310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470ea45-e7dd-4b7d-8cf8-23af7e8dcac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:32.891916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470ea45-e7dd-4b7d-8cf8-23af7e8dcac1 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:32.892197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470ea45-e7dd-4b7d-8cf8-23af7e8dcac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:32.892474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470ea45-e7dd-4b7d-8cf8-23af7e8dcac1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:32.893617 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470ea45-e7dd-4b7d-8cf8-23af7e8dcac1 None admin] RBAC: Adding query filter params (interface=admin, service_id=bb5e935613cd46f38c3dadefdfe185c1, region_id=RegionOne) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:32.894096 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470ea45-e7dd-4b7d-8cf8-23af7e8dcac1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:32.895459 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470ea45-e7dd-4b7d-8cf8-23af7e8dcac1 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:32.900534 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 497/984] 10.222.0.60 () {56 vars in 1249 bytes} [Wed Sep 28 12:10:32 2022] GET /identity/v3/endpoints?service_id=bb5e935613cd46f38c3dadefdfe185c1&interface=admin®ion_id=RegionOne => generated 193 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:32.906768 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470ea45-e7dd-4b7d-8cf8-23af7e8dcac1 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:32.911862 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a546d-cb43-4a9b-8f6d-adf106aae7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:32.912467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a546d-cb43-4a9b-8f6d-adf106aae7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:32.912756 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a546d-cb43-4a9b-8f6d-adf106aae7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:32.913009 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a546d-cb43-4a9b-8f6d-adf106aae7a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:32.913677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a546d-cb43-4a9b-8f6d-adf106aae7a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:32.913977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a546d-cb43-4a9b-8f6d-adf106aae7a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:32.915331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a546d-cb43-4a9b-8f6d-adf106aae7a3 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:32.927807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 488/985] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:32.933159 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a546d-cb43-4a9b-8f6d-adf106aae7a3 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:32.975441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:33.022648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470ea45-e7dd-4b7d-8cf8-23af7e8dcac1 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:33.044341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a546d-cb43-4a9b-8f6d-adf106aae7a3 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:33.060517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d0454-8f5e-41b5-87ad-5c71eb054e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:33.061287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d0454-8f5e-41b5-87ad-5c71eb054e92 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:33.061609 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d0454-8f5e-41b5-87ad-5c71eb054e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:33.061876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d0454-8f5e-41b5-87ad-5c71eb054e92 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:33.063124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d0454-8f5e-41b5-87ad-5c71eb054e92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:33.063967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d0454-8f5e-41b5-87ad-5c71eb054e92 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:33.070625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 498/986] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 12:10:32 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:10:33.088875 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd789e-6e15-48ec-a3a9-aa730fae3751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:33.089584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd789e-6e15-48ec-a3a9-aa730fae3751 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:33.089865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd789e-6e15-48ec-a3a9-aa730fae3751 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:33.090138 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd789e-6e15-48ec-a3a9-aa730fae3751 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:33.098745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd789e-6e15-48ec-a3a9-aa730fae3751 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:33.100237 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd789e-6e15-48ec-a3a9-aa730fae3751 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:33.113998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-a3fd789e-6e15-48ec-a3a9-aa730fae3751 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:33.113998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:33.113998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:33.113998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:33.113998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:33.113998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:33.113998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:33.113998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:33.113998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:33.113998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:33.113998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:33.113998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:33.113998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:10:33.113998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:10:33.113998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:10:33.114942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:10:33.114942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:33.114942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:33.114942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:33.114942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:33.114942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:33.114942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:33.114942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:33.114942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:33.114942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:33.114942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:33.114942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:33.114942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:33.114942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:33.114942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:33.114942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:33.114942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:33.121116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:10:33.121116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:10:33.121116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:10:33.121116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:10:33.121116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:10:33.121116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:10:33.121116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:33.121116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:33.121116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 489/987] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:33.122998 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d0454-8f5e-41b5-87ad-5c71eb054e92 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:33.240589 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d0454-8f5e-41b5-87ad-5c71eb054e92 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:33.267672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:33.276164 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9329d1e-ed82-4877-b763-9b5e18fe9ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:33.276772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9329d1e-ed82-4877-b763-9b5e18fe9ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:33.277053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9329d1e-ed82-4877-b763-9b5e18fe9ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:33.277361 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9329d1e-ed82-4877-b763-9b5e18fe9ac2 None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:33.278046 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9329d1e-ed82-4877-b763-9b5e18fe9ac2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:33.278350 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9329d1e-ed82-4877-b763-9b5e18fe9ac2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:33.279764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9329d1e-ed82-4877-b763-9b5e18fe9ac2 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:33.292607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 499/988] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:10:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:33.298937 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd789e-6e15-48ec-a3a9-aa730fae3751 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:33.308247 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:33.404159 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd789e-6e15-48ec-a3a9-aa730fae3751 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:33.438272 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883a403-7a6f-4e0b-8595-9faa51f7b13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:33.438881 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883a403-7a6f-4e0b-8595-9faa51f7b13e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:33.439163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883a403-7a6f-4e0b-8595-9faa51f7b13e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:33.439434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883a403-7a6f-4e0b-8595-9faa51f7b13e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:33.440156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883a403-7a6f-4e0b-8595-9faa51f7b13e None admin] RBAC: Adding query filter params (role.id=8af05c40a1604332a08a2c6ba7ad6de7, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=bc69cf267bea41bf890bc7551d84ad6f) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:33.440463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883a403-7a6f-4e0b-8595-9faa51f7b13e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:33.441911 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883a403-7a6f-4e0b-8595-9faa51f7b13e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:33.453163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 490/989] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:10:33 2022] GET /identity/v3/role_assignments?user.id=bc69cf267bea41bf890bc7551d84ad6f&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=8af05c40a1604332a08a2c6ba7ad6de7 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:34.026666 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb482507-eebb-4a47-858d-e2f233b9e788 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:34.027200 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb482507-eebb-4a47-858d-e2f233b9e788 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:34.027200 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb482507-eebb-4a47-858d-e2f233b9e788 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:34.028657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 500/990] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:34.034375 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4a0ec-ec03-4705-a6b6-96fb4659c4a5 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:34.034764 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4a0ec-ec03-4705-a6b6-96fb4659c4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:34.034942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4a0ec-ec03-4705-a6b6-96fb4659c4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:34.053152 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-4cb4a0ec-ec03-4705-a6b6-96fb4659c4a5 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:34.054321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-4cb4a0ec-ec03-4705-a6b6-96fb4659c4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:34.115885 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:34.121660 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 491/991] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:34.388252 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb482507-eebb-4a47-858d-e2f233b9e788 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:34.486808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb482507-eebb-4a47-858d-e2f233b9e788 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:34.519597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d43bf-8332-4ebc-ada8-211e5b428652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:34.520217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d43bf-8332-4ebc-ada8-211e5b428652 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:34.520490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d43bf-8332-4ebc-ada8-211e5b428652 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:34.520758 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449d43bf-8332-4ebc-ada8-211e5b428652 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:34.521357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d43bf-8332-4ebc-ada8-211e5b428652 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:34.522702 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449d43bf-8332-4ebc-ada8-211e5b428652 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:34.531880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-449d43bf-8332-4ebc-ada8-211e5b428652 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 12:10:34.531880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:34.531880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:34.531880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:34.531880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:34.531880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:34.531880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:34.531880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:34.531880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 12:10:34.531880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:34.531880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:34.531880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:34.531880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:34.531880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:34.531880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:34.531880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:34.531880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:34.532844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:34.532844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:34.532844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:34.532844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:34.532844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:34.532844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:34.532844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:34.532844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:34.532844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:34.532844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:34.532844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:34.532844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:34.532844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:34.532844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:34.532844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:34.532844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:34.533751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:34.533751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:34.533751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:34.533751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:34.533751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:34.533751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:34.533751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:34.533751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:34.533751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:34.533751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:34.533751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:34.533751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:34.533751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 12:10:34.533751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:34.535277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 501/992] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:34.538729 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4a0ec-ec03-4705-a6b6-96fb4659c4a5 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:34.636192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4a0ec-ec03-4705-a6b6-96fb4659c4a5 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:34.670352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73d1cf-8d92-4942-a0da-f00d3a79b636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:34.671660 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73d1cf-8d92-4942-a0da-f00d3a79b636 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:34.671942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73d1cf-8d92-4942-a0da-f00d3a79b636 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:34.672209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73d1cf-8d92-4942-a0da-f00d3a79b636 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:34.672825 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73d1cf-8d92-4942-a0da-f00d3a79b636 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:34.673125 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73d1cf-8d92-4942-a0da-f00d3a79b636 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:34.674522 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73d1cf-8d92-4942-a0da-f00d3a79b636 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:34.684327 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 492/993] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 12:10:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:34.685312 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d43bf-8332-4ebc-ada8-211e5b428652 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:34.739387 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:34.787701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449d43bf-8332-4ebc-ada8-211e5b428652 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:34.821632 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4ab4c-d343-451b-b93f-f6ee4cf45d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:34.822238 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4ab4c-d343-451b-b93f-f6ee4cf45d67 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:34.822516 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4ab4c-d343-451b-b93f-f6ee4cf45d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:34.822786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4ab4c-d343-451b-b93f-f6ee4cf45d67 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:34.823402 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4ab4c-d343-451b-b93f-f6ee4cf45d67 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:34.823701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4ab4c-d343-451b-b93f-f6ee4cf45d67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:34.825055 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4ab4c-d343-451b-b93f-f6ee4cf45d67 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:34.835435 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 502/994] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 12:10:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:34.836386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73d1cf-8d92-4942-a0da-f00d3a79b636 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:34.899123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:34.938197 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73d1cf-8d92-4942-a0da-f00d3a79b636 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:34.972121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb1712-899d-4a28-90c8-1411ea4aa7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:34.972727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb1712-899d-4a28-90c8-1411ea4aa7e3 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:34.973015 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb1712-899d-4a28-90c8-1411ea4aa7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:34.973314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb1712-899d-4a28-90c8-1411ea4aa7e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:34.974005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb1712-899d-4a28-90c8-1411ea4aa7e3 None admin] RBAC: Adding query filter params (interface=public, service_id=a2cc036c911b43b0a633f8b58a00de2f, region_id=RegionOne) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:34.974312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb1712-899d-4a28-90c8-1411ea4aa7e3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:34.975675 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fb1712-899d-4a28-90c8-1411ea4aa7e3 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:34.980779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 493/995] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Sep 28 12:10:34 2022] GET /identity/v3/endpoints?service_id=a2cc036c911b43b0a633f8b58a00de2f&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:34.987170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4ab4c-d343-451b-b93f-f6ee4cf45d67 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:35.086943 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4ab4c-d343-451b-b93f-f6ee4cf45d67 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:35.103862 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:35.126847 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c6fc6-c9f8-4aaf-981c-d16789643ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:35.127478 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c6fc6-c9f8-4aaf-981c-d16789643ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:35.127757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c6fc6-c9f8-4aaf-981c-d16789643ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:35.128028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c6fc6-c9f8-4aaf-981c-d16789643ce4 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:35.128640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c6fc6-c9f8-4aaf-981c-d16789643ce4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:35.130083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c6fc6-c9f8-4aaf-981c-d16789643ce4 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:35.141182 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 503/996] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 12:10:34 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:10:35.229568 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bc714-f5f9-46ec-9b57-d2f164758ea5 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:35.230075 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bc714-f5f9-46ec-9b57-d2f164758ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:35.230075 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082bc714-f5f9-46ec-9b57-d2f164758ea5 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:35.231564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 494/997] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:35.238224 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f23e5e-6657-4d43-8b3b-cc4befbb495b None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:35.238519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f23e5e-6657-4d43-8b3b-cc4befbb495b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:35.238755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f23e5e-6657-4d43-8b3b-cc4befbb495b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:35.257603 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-76f23e5e-6657-4d43-8b3b-cc4befbb495b None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:35.258814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-76f23e5e-6657-4d43-8b3b-cc4befbb495b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:35.337245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 504/998] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:35.601924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082bc714-f5f9-46ec-9b57-d2f164758ea5 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:35.625274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244eac4d-df81-41c3-aef1-ec3b734c39cb None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:35.625562 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244eac4d-df81-41c3-aef1-ec3b734c39cb None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:35.625836 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244eac4d-df81-41c3-aef1-ec3b734c39cb None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:35.627392 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 505/999] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:35.632135 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ea0d8-5a9e-4d5a-9e35-fde298ceed4a None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:35.632414 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ea0d8-5a9e-4d5a-9e35-fde298ceed4a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:35.632695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ea0d8-5a9e-4d5a-9e35-fde298ceed4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:35.650889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-782ea0d8-5a9e-4d5a-9e35-fde298ceed4a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:35.652055 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-782ea0d8-5a9e-4d5a-9e35-fde298ceed4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:35.709619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082bc714-f5f9-46ec-9b57-d2f164758ea5 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:35.721423 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 506/1000] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:10:35.742296 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:35.750844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c566cd-51a3-477f-beab-2ad2f80250ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:35.751471 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c566cd-51a3-477f-beab-2ad2f80250ef None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:35.751747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c566cd-51a3-477f-beab-2ad2f80250ef None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:35.752016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c566cd-51a3-477f-beab-2ad2f80250ef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:35.752603 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c566cd-51a3-477f-beab-2ad2f80250ef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:35.753952 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c566cd-51a3-477f-beab-2ad2f80250ef None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:35.762646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-b3c566cd-51a3-477f-beab-2ad2f80250ef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 12:10:35.762646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:35.762646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:35.762646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:35.762646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:35.762646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:35.762646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:35.762646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:35.762646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 12:10:35.762646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:35.762646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:35.762646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:35.762646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:35.762646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:35.762646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:35.762646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:35.762646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:35.763589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:35.763589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:35.763589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:35.763589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:35.763589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:35.763589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:35.763589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:35.763589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:35.763589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:35.763589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:35.763589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:35.763589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:35.763589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:35.763589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:35.763589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:35.763589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:35.766136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:35.766136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:35.766136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:35.766136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:35.766136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:35.766136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:35.766136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:35.766136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:35.766136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:35.766136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:35.766136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:35.766136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:35.766136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 12:10:35.766136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:35.766136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 495/1001] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Sep 28 12:10:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:35.770603 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ea0d8-5a9e-4d5a-9e35-fde298ceed4a None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:35.813975 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:35.871795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ea0d8-5a9e-4d5a-9e35-fde298ceed4a None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:35.905952 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59931f-062d-4b8e-8085-e09f2049dcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:35.906691 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59931f-062d-4b8e-8085-e09f2049dcb0 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:35.906968 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59931f-062d-4b8e-8085-e09f2049dcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:35.907236 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59931f-062d-4b8e-8085-e09f2049dcb0 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:35.907850 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f59931f-062d-4b8e-8085-e09f2049dcb0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:35.908153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f59931f-062d-4b8e-8085-e09f2049dcb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:35.909522 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f59931f-062d-4b8e-8085-e09f2049dcb0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:35.914366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 507/1002] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 12:10:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:35.919925 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c566cd-51a3-477f-beab-2ad2f80250ef None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:35.991208 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59931f-062d-4b8e-8085-e09f2049dcb0 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:36.021811 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c566cd-51a3-477f-beab-2ad2f80250ef None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:36.057533 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db6bd0-3ad3-48fe-a791-1156d91f4924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:36.058149 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db6bd0-3ad3-48fe-a791-1156d91f4924 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:36.058437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db6bd0-3ad3-48fe-a791-1156d91f4924 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:36.058695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db6bd0-3ad3-48fe-a791-1156d91f4924 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:36.059322 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db6bd0-3ad3-48fe-a791-1156d91f4924 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:36.059621 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db6bd0-3ad3-48fe-a791-1156d91f4924 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:36.060985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db6bd0-3ad3-48fe-a791-1156d91f4924 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:36.066460 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 496/1003] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Sep 28 12:10:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:36.072226 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db6bd0-3ad3-48fe-a791-1156d91f4924 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:36.098119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59931f-062d-4b8e-8085-e09f2049dcb0 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:36.100393 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:36.130365 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:36.136480 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69569a-7734-45a5-b19e-1ddbf64459d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:36.137106 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69569a-7734-45a5-b19e-1ddbf64459d0 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:36.137409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69569a-7734-45a5-b19e-1ddbf64459d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:36.137679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69569a-7734-45a5-b19e-1ddbf64459d0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:36.138248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69569a-7734-45a5-b19e-1ddbf64459d0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:36.139714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf69569a-7734-45a5-b19e-1ddbf64459d0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:36.148786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-bf69569a-7734-45a5-b19e-1ddbf64459d0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 12:10:36.148786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:36.148786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:36.148786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:36.148786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:36.148786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:36.148786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:36.148786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:36.148786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 12:10:36.148786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:36.148786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:36.148786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:36.148786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:36.148786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:36.148786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:36.148786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:36.148786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:36.149887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:36.149887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:36.149887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:36.149887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:36.149887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:36.149887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:36.149887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:36.149887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:36.149887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:36.149887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:36.149887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:36.149887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:36.149887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:36.149887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:36.149887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:36.149887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:36.150763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:36.150763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:36.150763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:36.150763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:36.150763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:36.150763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:36.150763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:36.150763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:36.150763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:36.150763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:36.150763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:36.150763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:36.150763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 12:10:36.150763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:36.150763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 508/1004] 10.222.0.60 () {56 vars in 1105 bytes} [Wed Sep 28 12:10:35 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:36.156240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69569a-7734-45a5-b19e-1ddbf64459d0 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:36.185865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db6bd0-3ad3-48fe-a791-1156d91f4924 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:36.221810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10126dd2-3dad-4526-bbcf-8639d0a30ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:36.222442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10126dd2-3dad-4526-bbcf-8639d0a30ebe None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:36.222683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10126dd2-3dad-4526-bbcf-8639d0a30ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:36.222949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10126dd2-3dad-4526-bbcf-8639d0a30ebe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:36.223715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10126dd2-3dad-4526-bbcf-8639d0a30ebe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:36.224975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10126dd2-3dad-4526-bbcf-8639d0a30ebe None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:36.249014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 497/1005] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:10:36 2022] POST /identity/v3/endpoints => generated 325 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:10:36.296055 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69569a-7734-45a5-b19e-1ddbf64459d0 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:36.340630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee8ded9-7e02-4a8e-90e2-df7a7508622f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:36.341324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8ded9-7e02-4a8e-90e2-df7a7508622f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:36.341606 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8ded9-7e02-4a8e-90e2-df7a7508622f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:36.341876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8ded9-7e02-4a8e-90e2-df7a7508622f None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:36.342566 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8ded9-7e02-4a8e-90e2-df7a7508622f None admin] RBAC: Adding query filter params (name=image) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:36.343378 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8ded9-7e02-4a8e-90e2-df7a7508622f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:36.345052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee8ded9-7e02-4a8e-90e2-df7a7508622f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:36.350624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 509/1006] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:36.356198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10126dd2-3dad-4526-bbcf-8639d0a30ebe None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:36.466697 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10126dd2-3dad-4526-bbcf-8639d0a30ebe None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:36.484088 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:36.504295 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5fb88e-bff3-428e-84d0-3034fad288f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:36.504903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fb88e-bff3-428e-84d0-3034fad288f2 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:36.505216 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fb88e-bff3-428e-84d0-3034fad288f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:36.505490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fb88e-bff3-428e-84d0-3034fad288f2 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:36.506114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5fb88e-bff3-428e-84d0-3034fad288f2 None admin] RBAC: Adding query filter params (type=image) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:36.506413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5fb88e-bff3-428e-84d0-3034fad288f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:36.507771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5fb88e-bff3-428e-84d0-3034fad288f2 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:36.512403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 498/1007] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:37.347088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485ec7f-5a36-4481-a538-a984f6a6103e None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:37.347600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485ec7f-5a36-4481-a538-a984f6a6103e None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:37.347676 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485ec7f-5a36-4481-a538-a984f6a6103e None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:37.349109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 510/1008] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:37.354144 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd86ba-ae09-438d-8c59-9f7bf12c128a None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:37.354546 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd86ba-ae09-438d-8c59-9f7bf12c128a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:37.354727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd86ba-ae09-438d-8c59-9f7bf12c128a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:37.375303 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-d3dd86ba-ae09-438d-8c59-9f7bf12c128a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:37.376528 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d3dd86ba-ae09-438d-8c59-9f7bf12c128a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:37.446538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 499/1009] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:37 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:37.716823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485ec7f-5a36-4481-a538-a984f6a6103e None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:37.747161 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:37.822572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485ec7f-5a36-4481-a538-a984f6a6103e None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:37.856399 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc501a-3400-46fa-8147-016912c1cf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:37.857109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc501a-3400-46fa-8147-016912c1cf6c None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:37.857474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc501a-3400-46fa-8147-016912c1cf6c None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:37.857812 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc501a-3400-46fa-8147-016912c1cf6c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:37.858747 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc501a-3400-46fa-8147-016912c1cf6c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:37.860281 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc501a-3400-46fa-8147-016912c1cf6c None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:37.869375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-d4bc501a-3400-46fa-8147-016912c1cf6c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 12:10:37.869375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:37.869375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:37.869375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:37.869375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:37.869375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:37.869375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:37.869375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:37.869375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 12:10:37.869375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:37.869375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:37.869375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:37.869375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:37.869375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:37.869375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:37.869375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:37.869375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:37.870648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:37.870648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:37.870648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:37.870648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:37.870648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:37.870648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:37.870648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:37.870648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:37.870648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:37.870648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:37.870648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:37.870648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:37.870648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:37.870648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:37.870648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:37.870648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:37.871753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:37.871753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:37.871753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:37.871753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:37.871753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:37.871753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:37.871753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:37.871753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:37.871753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:37.871753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:37.871753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:37.871753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:37.871753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 12:10:37.871753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:37.873278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 511/1010] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:37.877952 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd86ba-ae09-438d-8c59-9f7bf12c128a None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:37.978652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd86ba-ae09-438d-8c59-9f7bf12c128a None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:37.989904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:38.015119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5bf724-f09b-488e-be28-7e9d9d5612bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:38.015843 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bf724-f09b-488e-be28-7e9d9d5612bf None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:38.016127 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bf724-f09b-488e-be28-7e9d9d5612bf None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:38.016399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bf724-f09b-488e-be28-7e9d9d5612bf None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:38.017024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5bf724-f09b-488e-be28-7e9d9d5612bf None admin] RBAC: Adding query filter params (name=compute) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:38.017357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5bf724-f09b-488e-be28-7e9d9d5612bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:38.018762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5bf724-f09b-488e-be28-7e9d9d5612bf None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:38.028539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 500/1011] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 12:10:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:38.029511 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc501a-3400-46fa-8147-016912c1cf6c None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:38.124339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:38.137748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc501a-3400-46fa-8147-016912c1cf6c None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:38.177252 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bda832-9237-45a8-bd7d-0da3b4895252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:38.178118 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bda832-9237-45a8-bd7d-0da3b4895252 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:38.178627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bda832-9237-45a8-bd7d-0da3b4895252 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:38.179115 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bda832-9237-45a8-bd7d-0da3b4895252 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:38.179964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bda832-9237-45a8-bd7d-0da3b4895252 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:38.180493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bda832-9237-45a8-bd7d-0da3b4895252 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:38.182100 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bda832-9237-45a8-bd7d-0da3b4895252 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:38.188542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 512/1012] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Sep 28 12:10:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:38.194277 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5bf724-f09b-488e-be28-7e9d9d5612bf None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:38.298847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5bf724-f09b-488e-be28-7e9d9d5612bf None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:38.338020 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d3750-5789-4ce5-8aae-2c044136c99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:38.338628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d3750-5789-4ce5-8aae-2c044136c99d None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:38.338906 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d3750-5789-4ce5-8aae-2c044136c99d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:38.339175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d3750-5789-4ce5-8aae-2c044136c99d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:38.339833 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d3750-5789-4ce5-8aae-2c044136c99d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:38.341217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d3750-5789-4ce5-8aae-2c044136c99d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:38.361393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 501/1013] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:10:38 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:10:38.416360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5000c-ccff-49cd-abaa-b9d221be70fa None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:38.416909 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5000c-ccff-49cd-abaa-b9d221be70fa None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:38.416909 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5000c-ccff-49cd-abaa-b9d221be70fa None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:38.418562 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 513/1014] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:38.424489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52e980-75db-4ffd-9021-285ac5ef256d None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:38.424790 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52e980-75db-4ffd-9021-285ac5ef256d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:38.425039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52e980-75db-4ffd-9021-285ac5ef256d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:38.445773 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-4e52e980-75db-4ffd-9021-285ac5ef256d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:38.447002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-4e52e980-75db-4ffd-9021-285ac5ef256d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:38.473703 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:38.536587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 502/1015] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:38 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:38.688872 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2aaf59-ff80-4b64-bb9c-4be6b0d3a7b9 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:38.689244 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2aaf59-ff80-4b64-bb9c-4be6b0d3a7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:38.689455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2aaf59-ff80-4b64-bb9c-4be6b0d3a7b9 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:38.690927 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 514/1016] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:38.695957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928801e9-0d64-4737-b769-9a9008a1cc2d None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:38.696244 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928801e9-0d64-4737-b769-9a9008a1cc2d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:38.696518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928801e9-0d64-4737-b769-9a9008a1cc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:38.714905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-928801e9-0d64-4737-b769-9a9008a1cc2d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:38.716077 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-928801e9-0d64-4737-b769-9a9008a1cc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:38.784006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 503/1017] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:38 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:38.807916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2aaf59-ff80-4b64-bb9c-4be6b0d3a7b9 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:38.912152 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2aaf59-ff80-4b64-bb9c-4be6b0d3a7b9 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:38.947216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecfbd6-5cda-44ce-a888-b83b8596e9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:38.947763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecfbd6-5cda-44ce-a888-b83b8596e9bb None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:38.948041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecfbd6-5cda-44ce-a888-b83b8596e9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:38.948312 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecfbd6-5cda-44ce-a888-b83b8596e9bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:38.949436 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ecfbd6-5cda-44ce-a888-b83b8596e9bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:38.951499 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ecfbd6-5cda-44ce-a888-b83b8596e9bb None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:38.952603 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 515/1018] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:38.958364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928801e9-0d64-4737-b769-9a9008a1cc2d None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:39.047735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:39.054071 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecfbd6-5cda-44ce-a888-b83b8596e9bb None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:39.066135 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928801e9-0d64-4737-b769-9a9008a1cc2d None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:39.085757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:39.102565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da230b-07c0-449b-8cbb-cebb0cd0f88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:39.103243 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da230b-07c0-449b-8cbb-cebb0cd0f88c None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:39.103534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da230b-07c0-449b-8cbb-cebb0cd0f88c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:39.103813 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da230b-07c0-449b-8cbb-cebb0cd0f88c None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:39.104494 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da230b-07c0-449b-8cbb-cebb0cd0f88c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:39.106000 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da230b-07c0-449b-8cbb-cebb0cd0f88c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:39.134946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 504/1019] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 12:10:38 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:10:39.186365 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecfbd6-5cda-44ce-a888-b83b8596e9bb None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:39.226287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225c87a-f877-400b-9c74-f3bbc9ea8ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:39.226811 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225c87a-f877-400b-9c74-f3bbc9ea8ad7 None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:39.227093 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225c87a-f877-400b-9c74-f3bbc9ea8ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:39.227364 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225c87a-f877-400b-9c74-f3bbc9ea8ad7 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:39.228033 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225c87a-f877-400b-9c74-f3bbc9ea8ad7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:39.229468 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225c87a-f877-400b-9c74-f3bbc9ea8ad7 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:39.253188 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 516/1020] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 12:10:39 2022] POST /identity/v3/services => generated 242 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:10:40.644314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c2df0-12dd-4eef-a426-265463d18f24 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:40.644839 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c2df0-12dd-4eef-a426-265463d18f24 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:40.644948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c2df0-12dd-4eef-a426-265463d18f24 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:40.646416 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 505/1021] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:40.651654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47c1d3-0b8e-4eae-ac20-165ea9dbfbc9 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:40.652070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47c1d3-0b8e-4eae-ac20-165ea9dbfbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:40.652234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47c1d3-0b8e-4eae-ac20-165ea9dbfbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:40.670370 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-fe47c1d3-0b8e-4eae-ac20-165ea9dbfbc9 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:40.672368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-fe47c1d3-0b8e-4eae-ac20-165ea9dbfbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:40.742066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 517/1022] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:41.007005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c2df0-12dd-4eef-a426-265463d18f24 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:41.110046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c2df0-12dd-4eef-a426-265463d18f24 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:41.144377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70f273-d5e8-4c2c-b84e-eeb0e9d3c8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:41.145018 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70f273-d5e8-4c2c-b84e-eeb0e9d3c8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:41.145318 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70f273-d5e8-4c2c-b84e-eeb0e9d3c8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:41.145593 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70f273-d5e8-4c2c-b84e-eeb0e9d3c8d2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:41.152748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:41.159067 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70f273-d5e8-4c2c-b84e-eeb0e9d3c8d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:10:41.164560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-7e70f273-d5e8-4c2c-b84e-eeb0e9d3c8d2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 12:10:41.164560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:41.164560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:41.164560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:41.164560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:41.164560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:41.164560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:41.164560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:41.164560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:41.164560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:41.164560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:41.164560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:41.164560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:10:41.164560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:10:41.164560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:10:41.164560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:10:41.165755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:10:41.165755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:10:41.165755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:10:41.165755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:10:41.165755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:41.165755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:41.165755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:41.165755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:41.165755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:41.165755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:41.165755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:41.165755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:41.165755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:41.165755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:41.165755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:41.165755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:41.166612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:41.166612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:41.166612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:41.166612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:41.166612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:10:41.166612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:10:41.166612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:10:41.166612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:10:41.166612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:10:41.166612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:10:41.166612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 12:10:41.166612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:41.173228 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 506/1023] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Sep 28 12:10:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:41.173950 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47c1d3-0b8e-4eae-ac20-165ea9dbfbc9 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:41.189089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:41.286775 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47c1d3-0b8e-4eae-ac20-165ea9dbfbc9 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:41.325594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427394fb-d415-4a9d-bc64-057e1175fe46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:41.326609 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427394fb-d415-4a9d-bc64-057e1175fe46 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:41.327590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427394fb-d415-4a9d-bc64-057e1175fe46 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:41.328144 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427394fb-d415-4a9d-bc64-057e1175fe46 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:41.328995 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427394fb-d415-4a9d-bc64-057e1175fe46 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:41.329479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427394fb-d415-4a9d-bc64-057e1175fe46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:41.331135 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427394fb-d415-4a9d-bc64-057e1175fe46 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:41.341153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 518/1024] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Sep 28 12:10:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:41.347203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70f273-d5e8-4c2c-b84e-eeb0e9d3c8d2 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:41.359260 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac83fd1-b9fe-40b2-a18f-348d48c73dab None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:41.359604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac83fd1-b9fe-40b2-a18f-348d48c73dab None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:41.359830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac83fd1-b9fe-40b2-a18f-348d48c73dab None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:41.361973 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 519/1025] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:10:41.367048 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3ae0d-15ee-42df-b028-b0424e1bcbe9 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:41.367388 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3ae0d-15ee-42df-b028-b0424e1bcbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:41.367665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3ae0d-15ee-42df-b028-b0424e1bcbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:41.386410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-08c3ae0d-15ee-42df-b028-b0424e1bcbe9 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:41.387638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-08c3ae0d-15ee-42df-b028-b0424e1bcbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:41.460556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 520/1026] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:10:41.461846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232392f6-aac3-412f-890a-820c2ac4aff4 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:41.462133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232392f6-aac3-412f-890a-820c2ac4aff4 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:41.462237 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70f273-d5e8-4c2c-b84e-eeb0e9d3c8d2 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:41.462410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232392f6-aac3-412f-890a-820c2ac4aff4 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:41.463666 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 521/1027] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:41.469173 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f6319-a300-4f29-a754-63678d1b5b58 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:41.469467 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f6319-a300-4f29-a754-63678d1b5b58 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:41.469747 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f6319-a300-4f29-a754-63678d1b5b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:41.477023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:41.491452 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-340f6319-a300-4f29-a754-63678d1b5b58 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:41.492668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-340f6319-a300-4f29-a754-63678d1b5b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:41.502376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2c897-a893-4768-958e-eb81f6be0979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:41.503011 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2c897-a893-4768-958e-eb81f6be0979 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:41.503291 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2c897-a893-4768-958e-eb81f6be0979 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:41.503561 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2c897-a893-4768-958e-eb81f6be0979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:41.515631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2c897-a893-4768-958e-eb81f6be0979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:41.517060 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2c897-a893-4768-958e-eb81f6be0979 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:41.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-24f2c897-a893-4768-958e-eb81f6be0979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:41.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:41.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:41.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:41.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:41.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:41.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:41.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:41.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:41.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:41.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:41.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:41.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:41.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:41.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:41.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:41.532647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:41.532647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:41.532647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:41.532647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:41.532647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:41.532647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:41.532647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:41.532647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:41.532647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:41.532647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:41.532647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:41.532647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:41.532647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:41.532647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:41.532647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:41.532647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:41.532647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:41.533835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:41.533835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:41.533835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:41.537463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 507/1028] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:41.538226 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2c897-a893-4768-958e-eb81f6be0979 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:41.567694 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 522/1029] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:10:41.596802 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:41.647593 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2c897-a893-4768-958e-eb81f6be0979 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:41.680814 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1e5bb-597e-409d-94fe-c254bf4e5243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:41.681436 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1e5bb-597e-409d-94fe-c254bf4e5243 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:41.681714 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1e5bb-597e-409d-94fe-c254bf4e5243 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:41.681983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be1e5bb-597e-409d-94fe-c254bf4e5243 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:41.682616 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1e5bb-597e-409d-94fe-c254bf4e5243 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:41.682913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1e5bb-597e-409d-94fe-c254bf4e5243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:41.684259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be1e5bb-597e-409d-94fe-c254bf4e5243 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:41.696724 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 508/1030] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:41.702232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f6319-a300-4f29-a754-63678d1b5b58 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:41.739170 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1e5bb-597e-409d-94fe-c254bf4e5243 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:41.802407 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f6319-a300-4f29-a754-63678d1b5b58 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:41.831635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:41.838124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845e90c-04a5-4bf6-8dda-2f00f82fd327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:41.838754 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845e90c-04a5-4bf6-8dda-2f00f82fd327 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:41.839027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845e90c-04a5-4bf6-8dda-2f00f82fd327 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:41.839294 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845e90c-04a5-4bf6-8dda-2f00f82fd327 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:41.846680 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845e90c-04a5-4bf6-8dda-2f00f82fd327 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:41.848108 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4845e90c-04a5-4bf6-8dda-2f00f82fd327 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:41.848286 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be1e5bb-597e-409d-94fe-c254bf4e5243 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:41.860060 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:41.860195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-4845e90c-04a5-4bf6-8dda-2f00f82fd327 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:10:41.860195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:41.860195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:41.860195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:41.860195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:41.860195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:41.860195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:41.860195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:41.860195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:41.860195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:41.860195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:41.860195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:41.860195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:10:41.860195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:10:41.860195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:10:41.860195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:41.861512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:41.863021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:41.863021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:41.863021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:10:41.863021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:10:41.863021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:10:41.863021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:10:41.863021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:10:41.863021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:10:41.863021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:10:41.863021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:41.865861 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 523/1031] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 12:10:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:41.866435 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845e90c-04a5-4bf6-8dda-2f00f82fd327 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:41.886575 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121b39d-848b-4de1-9e35-8bc1f7b37116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:41.887215 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121b39d-848b-4de1-9e35-8bc1f7b37116 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:41.887499 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121b39d-848b-4de1-9e35-8bc1f7b37116 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:41.887776 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121b39d-848b-4de1-9e35-8bc1f7b37116 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:41.888918 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121b39d-848b-4de1-9e35-8bc1f7b37116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:41.890412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121b39d-848b-4de1-9e35-8bc1f7b37116 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:41.891977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 509/1032] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:41.892728 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121b39d-848b-4de1-9e35-8bc1f7b37116 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:41.963056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845e90c-04a5-4bf6-8dda-2f00f82fd327 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:41.991479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121b39d-848b-4de1-9e35-8bc1f7b37116 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:41.996945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c04dca-6052-431a-97fd-0715443f5cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:41.997601 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c04dca-6052-431a-97fd-0715443f5cfb None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:41.997882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c04dca-6052-431a-97fd-0715443f5cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:41.998154 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c04dca-6052-431a-97fd-0715443f5cfb None admin] PATH_INFO: `/v3/services/image` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:41.999472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c04dca-6052-431a-97fd-0715443f5cfb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:42.000836 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c04dca-6052-431a-97fd-0715443f5cfb None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:42.009569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-55c04dca-6052-431a-97fd-0715443f5cfb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 12:10:42.009569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:42.009569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:42.009569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:42.009569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:42.009569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:42.009569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:42.009569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:42.009569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 12:10:42.009569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:42.009569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:42.009569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:42.009569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:42.009569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:42.009569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:42.009569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:42.009569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:42.010755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:42.010755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:42.010755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:42.010755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:42.010755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:42.010755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:42.010755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:42.010755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:42.010755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:42.010755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:42.010755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:42.010755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:42.010755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:42.010755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:42.010755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:42.010755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:42.011625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:42.011625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:42.011625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:42.011625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:42.011625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:42.011625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:42.011625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:42.011625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:42.011625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:42.011625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:42.011625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:42.011625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:42.011625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 12:10:42.011625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:42.011625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 524/1033] 10.222.0.60 () {56 vars in 1105 bytes} [Wed Sep 28 12:10:41 2022] GET /identity/v3/services/image => generated 86 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:42.012442 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c04dca-6052-431a-97fd-0715443f5cfb None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:42.031107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e338d-1cc8-4736-9ae6-b9409b4ca41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:42.031709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e338d-1cc8-4736-9ae6-b9409b4ca41c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:42.031985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e338d-1cc8-4736-9ae6-b9409b4ca41c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:42.032251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e338d-1cc8-4736-9ae6-b9409b4ca41c None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:42.032965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e338d-1cc8-4736-9ae6-b9409b4ca41c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:42.033319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e338d-1cc8-4736-9ae6-b9409b4ca41c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:42.034721 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e338d-1cc8-4736-9ae6-b9409b4ca41c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:42.051957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 510/1034] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 12:10:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:42.052800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e338d-1cc8-4736-9ae6-b9409b4ca41c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:42.071159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:42.085818 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:42.116807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c04dca-6052-431a-97fd-0715443f5cfb None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:42.150475 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f3759-10ba-42be-a71d-f2f997e667c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:42.151030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e338d-1cc8-4736-9ae6-b9409b4ca41c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:42.151139 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f3759-10ba-42be-a71d-f2f997e667c3 None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:42.151351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f3759-10ba-42be-a71d-f2f997e667c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:42.151625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f3759-10ba-42be-a71d-f2f997e667c3 None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:42.152266 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f3759-10ba-42be-a71d-f2f997e667c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:42.153766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f3759-10ba-42be-a71d-f2f997e667c3 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:42.176919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 525/1035] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:10:42 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:10:42.177801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f3759-10ba-42be-a71d-f2f997e667c3 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:42.199976 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4ce59-8611-437f-911f-496281a3119f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:42.200874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4ce59-8611-437f-911f-496281a3119f None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:42.201331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4ce59-8611-437f-911f-496281a3119f None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:42.201753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4ce59-8611-437f-911f-496281a3119f None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:42.202416 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4ce59-8611-437f-911f-496281a3119f None admin] RBAC: Adding query filter params (name=image) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:42.202728 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4ce59-8611-437f-911f-496281a3119f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:42.204174 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4ce59-8611-437f-911f-496281a3119f None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:42.214669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 511/1036] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:42.216469 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4ce59-8611-437f-911f-496281a3119f None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:42.299305 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f3759-10ba-42be-a71d-f2f997e667c3 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:42.316195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:42.335803 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:42.338959 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4ce59-8611-437f-911f-496281a3119f None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:42.346161 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18a9a7-2e79-484c-bf3e-475c521a5630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:42.347429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18a9a7-2e79-484c-bf3e-475c521a5630 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:42.347704 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18a9a7-2e79-484c-bf3e-475c521a5630 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:42.347967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18a9a7-2e79-484c-bf3e-475c521a5630 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:42.380307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f4744-f9b3-4689-9335-b9039ed543f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:42.381081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f4744-f9b3-4689-9335-b9039ed543f6 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:42.381390 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f4744-f9b3-4689-9335-b9039ed543f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:42.381665 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f4744-f9b3-4689-9335-b9039ed543f6 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:42.382817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f4744-f9b3-4689-9335-b9039ed543f6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:42.382817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f4744-f9b3-4689-9335-b9039ed543f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:42.384698 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f4744-f9b3-4689-9335-b9039ed543f6 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:42.390799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 512/1037] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:42.396637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f4744-f9b3-4689-9335-b9039ed543f6 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:42.431963 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18a9a7-2e79-484c-bf3e-475c521a5630 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:42.433436 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18a9a7-2e79-484c-bf3e-475c521a5630 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:42.511273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f4744-f9b3-4689-9335-b9039ed543f6 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:42.541182 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 526/1038] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:10:42 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:10:42.545936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18a9a7-2e79-484c-bf3e-475c521a5630 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:42.549740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95c708-3b2b-4d0d-a29f-189a437749a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:42.550363 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95c708-3b2b-4d0d-a29f-189a437749a2 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:42.550646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95c708-3b2b-4d0d-a29f-189a437749a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:42.550923 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b95c708-3b2b-4d0d-a29f-189a437749a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:42.551632 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95c708-3b2b-4d0d-a29f-189a437749a2 None admin] RBAC: Adding query filter params (interface=public, service_id=83fda109bfa04a04ae750a4ad1b2b76f, region_id=RegionOne) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:42.551944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95c708-3b2b-4d0d-a29f-189a437749a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:42.553337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b95c708-3b2b-4d0d-a29f-189a437749a2 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:42.563578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 513/1039] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Sep 28 12:10:42 2022] GET /identity/v3/endpoints?service_id=83fda109bfa04a04ae750a4ad1b2b76f&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:42.564588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:42.564883 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95c708-3b2b-4d0d-a29f-189a437749a2 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:42.595535 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:42.660595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18a9a7-2e79-484c-bf3e-475c521a5630 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:42.672501 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b95c708-3b2b-4d0d-a29f-189a437749a2 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:42.699535 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d370e-b638-4ae6-a5b7-effb60df7010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:42.700299 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d370e-b638-4ae6-a5b7-effb60df7010 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:42.700556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d370e-b638-4ae6-a5b7-effb60df7010 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:42.700819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d370e-b638-4ae6-a5b7-effb60df7010 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:42.710219 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d673d1f-3312-449b-a0de-746ecdb49201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:42.710832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d673d1f-3312-449b-a0de-746ecdb49201 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:42.711116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d673d1f-3312-449b-a0de-746ecdb49201 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:42.711393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d673d1f-3312-449b-a0de-746ecdb49201 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:42.711998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d673d1f-3312-449b-a0de-746ecdb49201 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:42.712393 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d370e-b638-4ae6-a5b7-effb60df7010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:42.713461 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d673d1f-3312-449b-a0de-746ecdb49201 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:42.713850 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d370e-b638-4ae6-a5b7-effb60df7010 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:42.720517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 514/1040] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 12:10:42 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:10:42.729080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-f30d370e-b638-4ae6-a5b7-effb60df7010 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:42.729080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:42.729080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:42.729080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:42.729080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:42.729080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:42.729080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:42.729080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:42.729080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:42.729080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:42.729080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:42.729080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:42.729080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:42.729080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:42.729080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:42.729080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:42.730292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:42.730292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:42.730292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:42.730292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:42.730292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:42.730292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:42.730292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:42.730292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:42.730292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:42.730292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:42.730292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:42.730292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:42.730292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:42.730292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:42.730292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:42.730292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:42.730292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:42.731359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:42.731359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:42.731359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:42.731359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 527/1041] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:42.736642 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d673d1f-3312-449b-a0de-746ecdb49201 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:42.841297 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:42.851752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d673d1f-3312-449b-a0de-746ecdb49201 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:42.890336 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f99d47-0945-43bd-a2bf-55dd787adb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:42.890964 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f99d47-0945-43bd-a2bf-55dd787adb82 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:42.891248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f99d47-0945-43bd-a2bf-55dd787adb82 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:42.891524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f99d47-0945-43bd-a2bf-55dd787adb82 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:42.892181 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f99d47-0945-43bd-a2bf-55dd787adb82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:42.892487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f99d47-0945-43bd-a2bf-55dd787adb82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:42.893889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f99d47-0945-43bd-a2bf-55dd787adb82 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:42.906768 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 515/1042] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:42.912612 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d370e-b638-4ae6-a5b7-effb60df7010 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:42.979646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:43.021036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d370e-b638-4ae6-a5b7-effb60df7010 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:43.058482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d9676-6754-43a5-b087-51f6a0bf6de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:43.059114 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d9676-6754-43a5-b087-51f6a0bf6de0 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:43.059394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d9676-6754-43a5-b087-51f6a0bf6de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:43.059668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d9676-6754-43a5-b087-51f6a0bf6de0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:43.066720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d9676-6754-43a5-b087-51f6a0bf6de0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:43.068155 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d9676-6754-43a5-b087-51f6a0bf6de0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:43.079981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-6f1d9676-6754-43a5-b087-51f6a0bf6de0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:43.079981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:43.079981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:43.079981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:43.079981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:43.079981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:43.079981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:43.079981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:43.079981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:43.079981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:43.079981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:43.079981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:43.079981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:10:43.079981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:10:43.079981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:10:43.080953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:10:43.080953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:43.080953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:43.080953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:43.080953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:43.080953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:43.080953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:43.080953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:43.080953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:43.080953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:43.080953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:43.080953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:43.080953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:43.080953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:43.080953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:43.080953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:43.080953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:43.081905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:10:43.081905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:10:43.081905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:10:43.081905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:10:43.081905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:10:43.081905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:10:43.081905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:43.081905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:43.081905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 528/1043] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:43.086863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f99d47-0945-43bd-a2bf-55dd787adb82 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:43.219316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f99d47-0945-43bd-a2bf-55dd787adb82 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:43.254747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fca19e-c4f0-42b1-9e5a-22636f719a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:43.255677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fca19e-c4f0-42b1-9e5a-22636f719a52 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:43.256181 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fca19e-c4f0-42b1-9e5a-22636f719a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:43.256708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fca19e-c4f0-42b1-9e5a-22636f719a52 None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:43.257614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fca19e-c4f0-42b1-9e5a-22636f719a52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:43.258128 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fca19e-c4f0-42b1-9e5a-22636f719a52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:43.259747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fca19e-c4f0-42b1-9e5a-22636f719a52 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:43.273008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 516/1044] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:10:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:43.278712 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d9676-6754-43a5-b087-51f6a0bf6de0 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:43.379344 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d9676-6754-43a5-b087-51f6a0bf6de0 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:43.402510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:43.414728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb0ded-8b8a-442a-962b-3fc455808cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:43.415333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb0ded-8b8a-442a-962b-3fc455808cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:43.415608 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb0ded-8b8a-442a-962b-3fc455808cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:43.415878 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb0ded-8b8a-442a-962b-3fc455808cd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:43.416650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb0ded-8b8a-442a-962b-3fc455808cd4 None admin] RBAC: Adding query filter params (role.id=8af05c40a1604332a08a2c6ba7ad6de7, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=982b82e8ac424318a934f9149fb91706) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:43.416960 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb0ded-8b8a-442a-962b-3fc455808cd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:43.418440 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb0ded-8b8a-442a-962b-3fc455808cd4 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:43.429241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 529/1045] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:10:43 2022] GET /identity/v3/role_assignments?user.id=982b82e8ac424318a934f9149fb91706&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=8af05c40a1604332a08a2c6ba7ad6de7 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:44.410761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fae335-f6a9-4b9a-9e09-587f4ae5dd27 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:44.411285 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fae335-f6a9-4b9a-9e09-587f4ae5dd27 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:44.411285 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fae335-f6a9-4b9a-9e09-587f4ae5dd27 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:44.412791 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 517/1046] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:44.417927 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e522cfb-e6f2-40dc-8f45-d46a1c8474f3 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:44.418310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e522cfb-e6f2-40dc-8f45-d46a1c8474f3 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:44.418466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e522cfb-e6f2-40dc-8f45-d46a1c8474f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:44.436948 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7e522cfb-e6f2-40dc-8f45-d46a1c8474f3 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:44.438230 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7e522cfb-e6f2-40dc-8f45-d46a1c8474f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:44.503787 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 530/1047] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:44.768996 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fae335-f6a9-4b9a-9e09-587f4ae5dd27 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:44.784524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:44.880282 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fae335-f6a9-4b9a-9e09-587f4ae5dd27 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:44.918677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb389e-7907-40fe-a164-421eb1fd4f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:44.919594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb389e-7907-40fe-a164-421eb1fd4f9a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:44.920026 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb389e-7907-40fe-a164-421eb1fd4f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:44.920486 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb389e-7907-40fe-a164-421eb1fd4f9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:44.930977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb389e-7907-40fe-a164-421eb1fd4f9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:10:44.936837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-adfb389e-7907-40fe-a164-421eb1fd4f9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:10:44.936837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:44.936837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:44.936837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:44.936837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:44.936837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:44.936837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:44.936837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:44.936837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:44.936837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:44.936837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:44.936837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:44.936837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:10:44.936837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:10:44.936837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:10:44.936837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:10:44.937873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:10:44.937873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:10:44.937873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:10:44.937873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:10:44.937873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:44.937873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:44.937873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:44.937873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:44.937873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:44.937873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:44.937873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:44.937873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:44.937873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:44.937873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:44.937873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:44.937873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:44.938745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:44.938745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:44.938745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:44.938745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:44.938745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:10:44.938745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:10:44.938745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:10:44.938745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:10:44.938745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:10:44.938745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:10:44.938745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:10:44.938745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:44.941284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 518/1048] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:10:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:44.944802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e522cfb-e6f2-40dc-8f45-d46a1c8474f3 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:44.987753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947c652-1b5f-42a3-9f33-2d71cd36159b None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:44.988056 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947c652-1b5f-42a3-9f33-2d71cd36159b None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:44.988312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947c652-1b5f-42a3-9f33-2d71cd36159b None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:44.991140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 519/1049] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:44.996101 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734553a-bf0f-474d-89dc-405737e8620e None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:44.996391 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734553a-bf0f-474d-89dc-405737e8620e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:44.996659 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734553a-bf0f-474d-89dc-405737e8620e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:45.020633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-d734553a-bf0f-474d-89dc-405737e8620e None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:45.021786 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d734553a-bf0f-474d-89dc-405737e8620e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:45.053979 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e522cfb-e6f2-40dc-8f45-d46a1c8474f3 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:45.073986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:45.093724 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 520/1050] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:10:45.094262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143bfc7-e6fe-4556-8304-f4e27d45451c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:45.094976 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143bfc7-e6fe-4556-8304-f4e27d45451c None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:45.095260 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143bfc7-e6fe-4556-8304-f4e27d45451c None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:45.095535 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143bfc7-e6fe-4556-8304-f4e27d45451c None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:45.096196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143bfc7-e6fe-4556-8304-f4e27d45451c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:45.096499 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143bfc7-e6fe-4556-8304-f4e27d45451c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:45.097914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143bfc7-e6fe-4556-8304-f4e27d45451c None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:45.111785 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 531/1051] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:10:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:45.113400 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734553a-bf0f-474d-89dc-405737e8620e None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:45.116099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:45.218206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734553a-bf0f-474d-89dc-405737e8620e None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:45.253183 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1de6c-f54a-4df4-96c8-1fd3b4290052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:45.253825 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1de6c-f54a-4df4-96c8-1fd3b4290052 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:45.254103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1de6c-f54a-4df4-96c8-1fd3b4290052 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:45.254375 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1de6c-f54a-4df4-96c8-1fd3b4290052 None admin] PATH_INFO: `/v3/users/dc6626d1a6bd42ef82f015a89d171065` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:45.268519 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1de6c-f54a-4df4-96c8-1fd3b4290052 None admin] RBAC: Authorizing `identity:get_user(user_id=dc6626d1a6bd42ef82f015a89d171065)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:45.270106 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1de6c-f54a-4df4-96c8-1fd3b4290052 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:45.271911 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 521/1052] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:10:45 2022] GET /identity/v3/users/dc6626d1a6bd42ef82f015a89d171065 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:45.277396 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143bfc7-e6fe-4556-8304-f4e27d45451c None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:45.363352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1de6c-f54a-4df4-96c8-1fd3b4290052 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:45.381759 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143bfc7-e6fe-4556-8304-f4e27d45451c None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:45.418012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d68c5-9b2f-4bdf-95d3-915b00b8d9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:45.418677 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d68c5-9b2f-4bdf-95d3-915b00b8d9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:45.418889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d68c5-9b2f-4bdf-95d3-915b00b8d9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:45.419157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d68c5-9b2f-4bdf-95d3-915b00b8d9c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:45.424590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:45.474921 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1de6c-f54a-4df4-96c8-1fd3b4290052 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:45.481020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:45.508877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d68c5-9b2f-4bdf-95d3-915b00b8d9c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:45.511367 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d68c5-9b2f-4bdf-95d3-915b00b8d9c0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:45.514268 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0a1d7-7a4d-463e-abde-17eaf7dd6125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:45.515238 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0a1d7-7a4d-463e-abde-17eaf7dd6125 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:45.515540 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0a1d7-7a4d-463e-abde-17eaf7dd6125 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:45.515815 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0a1d7-7a4d-463e-abde-17eaf7dd6125 None admin] PATH_INFO: `/v3/services/image` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:45.517079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0a1d7-7a4d-463e-abde-17eaf7dd6125 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:45.517987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0a1d7-7a4d-463e-abde-17eaf7dd6125 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:45.527073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-d3e0a1d7-7a4d-463e-abde-17eaf7dd6125 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 12:10:45.527073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:45.527073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:10:45.527073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:10:45.527073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:10:45.527073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:10:45.527073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:10:45.527073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:45.527073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 12:10:45.527073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:45.527073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:10:45.527073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:45.527073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:45.527073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:45.527073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:45.527073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:45.527073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:45.528041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:45.528041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:45.528041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:45.528041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:45.528041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:45.528041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:45.528041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:10:45.528041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:10:45.528041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:10:45.528041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:10:45.528041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:45.528041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:45.528041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:45.528041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:45.528041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:45.528041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:45.529630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:45.529630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:45.529630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:45.529630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:45.529630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:45.529630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:45.529630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:45.529630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:45.529630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:45.529630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:45.529630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:10:45.529630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:10:45.529630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 12:10:45.529630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:45.529630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 522/1053] 10.222.0.60 () {56 vars in 1105 bytes} [Wed Sep 28 12:10:45 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:45.534045 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0a1d7-7a4d-463e-abde-17eaf7dd6125 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:45.619570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 532/1054] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:10:45 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:10:45.622509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728b085-76ca-49cc-970a-95fab6c1feb2 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:45.622911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728b085-76ca-49cc-970a-95fab6c1feb2 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:45.623192 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728b085-76ca-49cc-970a-95fab6c1feb2 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:45.625187 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 533/1055] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:45.625946 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728b085-76ca-49cc-970a-95fab6c1feb2 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:45.639091 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0a1d7-7a4d-463e-abde-17eaf7dd6125 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:45.662752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:45.674075 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a904d5a-d08e-4490-9c19-b0b301ab3113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:45.674683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a904d5a-d08e-4490-9c19-b0b301ab3113 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:45.674968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a904d5a-d08e-4490-9c19-b0b301ab3113 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:45.675244 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a904d5a-d08e-4490-9c19-b0b301ab3113 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:45.675877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a904d5a-d08e-4490-9c19-b0b301ab3113 None admin] RBAC: Adding query filter params (name=image) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:45.676185 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a904d5a-d08e-4490-9c19-b0b301ab3113 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:45.677595 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a904d5a-d08e-4490-9c19-b0b301ab3113 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:45.682373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 523/1056] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:45.683582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb61eca-3e25-4750-9b61-c7520ccb4c99 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:45.683873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb61eca-3e25-4750-9b61-c7520ccb4c99 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:45.684151 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb61eca-3e25-4750-9b61-c7520ccb4c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:45.702070 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-1eb61eca-3e25-4750-9b61-c7520ccb4c99 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:45.703259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1eb61eca-3e25-4750-9b61-c7520ccb4c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:45.707016 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:45.727866 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728b085-76ca-49cc-970a-95fab6c1feb2 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:45.761343 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39959969-d3df-4c71-a52a-800ae0b29109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:45.761968 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39959969-d3df-4c71-a52a-800ae0b29109 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:45.762244 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39959969-d3df-4c71-a52a-800ae0b29109 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:45.762514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39959969-d3df-4c71-a52a-800ae0b29109 None admin] PATH_INFO: `/v3/projects/a2cea9c48e8e4299aedfbccd5e4c29db` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:45.763659 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39959969-d3df-4c71-a52a-800ae0b29109 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cea9c48e8e4299aedfbccd5e4c29db)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:45.765256 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39959969-d3df-4c71-a52a-800ae0b29109 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:45.766879 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 534/1057] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:10:45 2022] GET /identity/v3/projects/a2cea9c48e8e4299aedfbccd5e4c29db => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:45.767631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39959969-d3df-4c71-a52a-800ae0b29109 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:45.770786 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 524/1058] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:45.773813 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb61eca-3e25-4750-9b61-c7520ccb4c99 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:45.874089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39959969-d3df-4c71-a52a-800ae0b29109 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:45.883709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb61eca-3e25-4750-9b61-c7520ccb4c99 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:45.902826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:45.915446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8998dbab-1bd2-4ea9-9161-5c0588fe6ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:45.916050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998dbab-1bd2-4ea9-9161-5c0588fe6ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:45.916329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998dbab-1bd2-4ea9-9161-5c0588fe6ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:45.916602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8998dbab-1bd2-4ea9-9161-5c0588fe6ef8 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:45.917248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8998dbab-1bd2-4ea9-9161-5c0588fe6ef8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:45.917549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8998dbab-1bd2-4ea9-9161-5c0588fe6ef8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:45.918903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8998dbab-1bd2-4ea9-9161-5c0588fe6ef8 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:45.922865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36a344-b95f-4706-8d2e-45f29f0ed49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:45.923672 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36a344-b95f-4706-8d2e-45f29f0ed49b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:45.924148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36a344-b95f-4706-8d2e-45f29f0ed49b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:45.924599 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36a344-b95f-4706-8d2e-45f29f0ed49b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:45.924870 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 535/1059] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:45.925706 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36a344-b95f-4706-8d2e-45f29f0ed49b None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=a2cea9c48e8e4299aedfbccd5e4c29db, user.id=dc6626d1a6bd42ef82f015a89d171065) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:45.926019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36a344-b95f-4706-8d2e-45f29f0ed49b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:45.927442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36a344-b95f-4706-8d2e-45f29f0ed49b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:45.930385 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8998dbab-1bd2-4ea9-9161-5c0588fe6ef8 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:45.941630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 525/1060] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:10:45 2022] GET /identity/v3/role_assignments?user.id=dc6626d1a6bd42ef82f015a89d171065&scope.project.id=a2cea9c48e8e4299aedfbccd5e4c29db&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:45.946381 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:46.041457 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8998dbab-1bd2-4ea9-9161-5c0588fe6ef8 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:46.049524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36a344-b95f-4706-8d2e-45f29f0ed49b None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:46.083744 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aadbda-42d4-4af0-885a-72450fbc5c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:46.084370 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aadbda-42d4-4af0-885a-72450fbc5c41 None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:46.084647 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aadbda-42d4-4af0-885a-72450fbc5c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:46.084914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aadbda-42d4-4af0-885a-72450fbc5c41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:46.085613 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aadbda-42d4-4af0-885a-72450fbc5c41 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:46.086991 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aadbda-42d4-4af0-885a-72450fbc5c41 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:46.111733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 536/1061] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:10:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:10:46.176461 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36a344-b95f-4706-8d2e-45f29f0ed49b None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:46.223109 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267c5fe-f8a2-4ce4-b710-438d9cd2f50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:46.223755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267c5fe-f8a2-4ce4-b710-438d9cd2f50f None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:46.224033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267c5fe-f8a2-4ce4-b710-438d9cd2f50f None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:46.224307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267c5fe-f8a2-4ce4-b710-438d9cd2f50f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:46.236544 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267c5fe-f8a2-4ce4-b710-438d9cd2f50f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:10:46.241608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-b267c5fe-f8a2-4ce4-b710-438d9cd2f50f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 12:10:46.241608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:46.241608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:46.241608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:46.241608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:46.241608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:46.241608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:46.241608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:46.241608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:46.241608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:46.241608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:46.241608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:46.241608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:10:46.241608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:10:46.241608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:10:46.241608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:10:46.242600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:10:46.242600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:10:46.242600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:10:46.242600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:10:46.242600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:46.242600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:46.242600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:46.242600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:46.242600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:46.242600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:46.242600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:46.242600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:46.242600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:46.242600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:46.242600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:46.242600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:46.244089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:46.244089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:46.244089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:46.244089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:46.244089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:10:46.244089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:10:46.244089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:10:46.244089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:10:46.244089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:10:46.244089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:10:46.244089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 12:10:46.244089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:46.244089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 526/1062] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Sep 28 12:10:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:46.249528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aadbda-42d4-4af0-885a-72450fbc5c41 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:46.327404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:46.364732 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aadbda-42d4-4af0-885a-72450fbc5c41 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:46.399251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511d128-3ae7-469a-aa90-3d7d13479332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:46.399862 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511d128-3ae7-469a-aa90-3d7d13479332 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:46.400136 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511d128-3ae7-469a-aa90-3d7d13479332 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:46.400403 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511d128-3ae7-469a-aa90-3d7d13479332 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:46.401050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511d128-3ae7-469a-aa90-3d7d13479332 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:46.401379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511d128-3ae7-469a-aa90-3d7d13479332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:46.402727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511d128-3ae7-469a-aa90-3d7d13479332 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:46.411928 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 537/1063] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Sep 28 12:10:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:46.417788 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267c5fe-f8a2-4ce4-b710-438d9cd2f50f None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:46.459411 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:46.521432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267c5fe-f8a2-4ce4-b710-438d9cd2f50f None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:46.555748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7fa0c-cb94-4c28-a19f-ca282f7826d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:46.556372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7fa0c-cb94-4c28-a19f-ca282f7826d7 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:46.556648 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7fa0c-cb94-4c28-a19f-ca282f7826d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:46.556916 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7fa0c-cb94-4c28-a19f-ca282f7826d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:46.567569 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7fa0c-cb94-4c28-a19f-ca282f7826d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:46.569066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7fa0c-cb94-4c28-a19f-ca282f7826d7 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:46.583554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-ada7fa0c-cb94-4c28-a19f-ca282f7826d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:46.583554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:46.583554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:46.583554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:46.583554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:46.583554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:46.583554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:46.583554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:46.583554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:46.583554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:46.583554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:46.583554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:46.583554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:46.583554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:46.583554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:46.583554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:46.584643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:46.584643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:46.584643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:46.584643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:46.584643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:46.584643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:46.584643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:46.584643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:46.584643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:46.584643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:46.584643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:46.584643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:46.584643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:46.584643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:46.584643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:46.584643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:46.584643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:46.585598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:46.585598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:46.585598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:46.585598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 527/1064] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:46.591631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511d128-3ae7-469a-aa90-3d7d13479332 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:46.693810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511d128-3ae7-469a-aa90-3d7d13479332 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:46.722078 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:46.728849 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080990d-71fe-4c7e-abda-9b5fcc72710e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:46.729483 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080990d-71fe-4c7e-abda-9b5fcc72710e None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:46.729762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080990d-71fe-4c7e-abda-9b5fcc72710e None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:46.730081 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080990d-71fe-4c7e-abda-9b5fcc72710e None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:46.730715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080990d-71fe-4c7e-abda-9b5fcc72710e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:46.731013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080990d-71fe-4c7e-abda-9b5fcc72710e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:46.732362 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080990d-71fe-4c7e-abda-9b5fcc72710e None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:46.749321 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 538/1065] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:46.751060 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7fa0c-cb94-4c28-a19f-ca282f7826d7 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:46.845999 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:46.856803 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7fa0c-cb94-4c28-a19f-ca282f7826d7 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:46.891338 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0abef-2fb9-43e8-b173-3c1d1af37095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:46.891970 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0abef-2fb9-43e8-b173-3c1d1af37095 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:46.892246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0abef-2fb9-43e8-b173-3c1d1af37095 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:46.892513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0abef-2fb9-43e8-b173-3c1d1af37095 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:46.899874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0abef-2fb9-43e8-b173-3c1d1af37095 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:46.901396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0abef-2fb9-43e8-b173-3c1d1af37095 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:46.912731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-6ef0abef-2fb9-43e8-b173-3c1d1af37095 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:10:46.912731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:46.912731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:46.912731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:46.912731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:46.912731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:46.912731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:46.912731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:46.912731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:46.912731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:46.912731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:46.912731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:46.912731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:10:46.912731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:10:46.912731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:10:46.912731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:46.913715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:46.919439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:46.919439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:46.919439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:10:46.919439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:10:46.919439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:10:46.919439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:10:46.919439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:10:46.919439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:10:46.919439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:10:46.919439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:46.919439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 528/1066] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 12:10:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:46.920064 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080990d-71fe-4c7e-abda-9b5fcc72710e None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:47.017845 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080990d-71fe-4c7e-abda-9b5fcc72710e None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:47.053546 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92ca2f-d0d9-494b-819b-61a84feb1e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:47.054156 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92ca2f-d0d9-494b-819b-61a84feb1e38 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:47.054431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92ca2f-d0d9-494b-819b-61a84feb1e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:47.054718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92ca2f-d0d9-494b-819b-61a84feb1e38 None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:47.055405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92ca2f-d0d9-494b-819b-61a84feb1e38 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:47.055703 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92ca2f-d0d9-494b-819b-61a84feb1e38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:47.057105 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92ca2f-d0d9-494b-819b-61a84feb1e38 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:47.079094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 539/1067] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 12:10:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:47.080223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0abef-2fb9-43e8-b173-3c1d1af37095 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:47.191415 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0abef-2fb9-43e8-b173-3c1d1af37095 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:47.225611 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357e353-1bd4-4914-b468-d4ce5972c761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:47.226237 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357e353-1bd4-4914-b468-d4ce5972c761 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:47.226515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357e353-1bd4-4914-b468-d4ce5972c761 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:47.226784 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357e353-1bd4-4914-b468-d4ce5972c761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:47.237481 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357e353-1bd4-4914-b468-d4ce5972c761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:47.238912 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357e353-1bd4-4914-b468-d4ce5972c761 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:47.253442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-9357e353-1bd4-4914-b468-d4ce5972c761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:47.253442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:47.253442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:47.253442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:47.253442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:47.253442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:47.253442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:47.253442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:47.253442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:47.253442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:47.253442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:47.253442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:47.253442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:47.253442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:47.253442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:47.253442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:47.254384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:47.254384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:47.254384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:47.254384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:47.254384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:47.254384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:47.254384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:47.254384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:47.254384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:47.254384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:47.254384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:47.254384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:47.254384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:47.254384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:47.254384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:47.254384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:47.254384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:47.260384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:47.260384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:47.260384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:47.260384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 529/1068] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:47.260625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92ca2f-d0d9-494b-819b-61a84feb1e38 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:47.311798 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:47.363057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92ca2f-d0d9-494b-819b-61a84feb1e38 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:47.397662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a254d0-783d-42eb-a19c-6b5105681d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:47.398269 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a254d0-783d-42eb-a19c-6b5105681d2a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:47.398550 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a254d0-783d-42eb-a19c-6b5105681d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:47.398823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a254d0-783d-42eb-a19c-6b5105681d2a None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:47.399457 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a254d0-783d-42eb-a19c-6b5105681d2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:47.399763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a254d0-783d-42eb-a19c-6b5105681d2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:47.401125 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a254d0-783d-42eb-a19c-6b5105681d2a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:47.412628 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 540/1069] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:47.418033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357e353-1bd4-4914-b468-d4ce5972c761 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:47.420916 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:47.520799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357e353-1bd4-4914-b468-d4ce5972c761 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:47.554374 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e9c66-e061-4669-9509-1ad275b91fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:47.555004 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e9c66-e061-4669-9509-1ad275b91fce None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:47.555285 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e9c66-e061-4669-9509-1ad275b91fce None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:47.555558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e9c66-e061-4669-9509-1ad275b91fce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:47.562287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e9c66-e061-4669-9509-1ad275b91fce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:47.563663 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e9c66-e061-4669-9509-1ad275b91fce None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:47.574917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-562e9c66-e061-4669-9509-1ad275b91fce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:47.574917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:47.574917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:47.574917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:47.574917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:47.574917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:47.574917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:47.574917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:47.574917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:47.574917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:47.574917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:47.574917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:47.574917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:10:47.574917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:10:47.574917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:10:47.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:10:47.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:47.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:47.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:47.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:47.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:47.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:47.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:47.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:47.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:47.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:47.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:47.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:47.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:47.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:47.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:47.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:47.577870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:10:47.577870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:10:47.577870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:10:47.577870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:10:47.577870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:10:47.577870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:10:47.577870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:47.577870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:47.582780 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 530/1070] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:47.582910 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a254d0-783d-42eb-a19c-6b5105681d2a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:47.685162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a254d0-783d-42eb-a19c-6b5105681d2a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:47.694100 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:47.720819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49191e-ad36-40df-ba05-3f3f233599c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:47.721458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49191e-ad36-40df-ba05-3f3f233599c1 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:47.721739 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49191e-ad36-40df-ba05-3f3f233599c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:47.722009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49191e-ad36-40df-ba05-3f3f233599c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:47.722693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49191e-ad36-40df-ba05-3f3f233599c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:47.723000 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49191e-ad36-40df-ba05-3f3f233599c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:47.724420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49191e-ad36-40df-ba05-3f3f233599c1 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:47.742043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 541/1071] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:10:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:47.743244 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e9c66-e061-4669-9509-1ad275b91fce None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:47.807039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:47.849457 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e9c66-e061-4669-9509-1ad275b91fce None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:47.882800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23ae86-dc26-46e4-bdfe-51595498828b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:47.883552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23ae86-dc26-46e4-bdfe-51595498828b None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:47.883901 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23ae86-dc26-46e4-bdfe-51595498828b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:47.884244 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23ae86-dc26-46e4-bdfe-51595498828b None admin] PATH_INFO: `/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/982b82e8ac424318a934f9149fb91706/roles/8af05c40a1604332a08a2c6ba7ad6de7` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:47.886466 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23ae86-dc26-46e4-bdfe-51595498828b None admin] RBAC: Authorizing `identity:create_grant(project_id=6335d0db1f4844c39c255909189e0ce9, user_id=982b82e8ac424318a934f9149fb91706, role_id=8af05c40a1604332a08a2c6ba7ad6de7)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:47.887964 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23ae86-dc26-46e4-bdfe-51595498828b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:47.902010 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 531/1072] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:10:47 2022] PUT /identity/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/982b82e8ac424318a934f9149fb91706/roles/8af05c40a1604332a08a2c6ba7ad6de7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:10:48.103680 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fb1d8-b297-4df1-9c7d-bd717c88f870 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:48.103876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fb1d8-b297-4df1-9c7d-bd717c88f870 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:48.104151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fb1d8-b297-4df1-9c7d-bd717c88f870 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:48.105659 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 542/1073] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:48.111620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48377a56-3d12-4cd0-bccd-876fd8c3b874 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:48.111903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48377a56-3d12-4cd0-bccd-876fd8c3b874 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:48.112174 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48377a56-3d12-4cd0-bccd-876fd8c3b874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:48.130184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-48377a56-3d12-4cd0-bccd-876fd8c3b874 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:48.131362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-48377a56-3d12-4cd0-bccd-876fd8c3b874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:48.213187 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 532/1074] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:48.318998 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d46f8-2bb2-4665-bb5e-4a996c589d46 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:48.319315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d46f8-2bb2-4665-bb5e-4a996c589d46 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:48.319545 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d46f8-2bb2-4665-bb5e-4a996c589d46 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:48.321015 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 543/1075] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:48.325989 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d061d37-da6e-4a80-801d-8352377c5185 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:48.326966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d061d37-da6e-4a80-801d-8352377c5185 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:48.327246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d061d37-da6e-4a80-801d-8352377c5185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:48.345863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-7d061d37-da6e-4a80-801d-8352377c5185 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:48.347089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7d061d37-da6e-4a80-801d-8352377c5185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:48.396110 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:48.417687 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 533/1076] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:48.489220 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d46f8-2bb2-4665-bb5e-4a996c589d46 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:48.594058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d46f8-2bb2-4665-bb5e-4a996c589d46 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:48.628310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1e1e3-5ef0-4088-b0f7-76f42439de97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:48.628945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1e1e3-5ef0-4088-b0f7-76f42439de97 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:48.629378 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1e1e3-5ef0-4088-b0f7-76f42439de97 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:48.629657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1e1e3-5ef0-4088-b0f7-76f42439de97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:48.640455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1e1e3-5ef0-4088-b0f7-76f42439de97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:10:48.645358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-3bd1e1e3-5ef0-4088-b0f7-76f42439de97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:10:48.645358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:48.645358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:48.645358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:48.645358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:48.645358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:48.645358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:48.645358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:48.645358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:48.645358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:48.645358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:48.645358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:48.645358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:10:48.645358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:10:48.645358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:10:48.645358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:10:48.646307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:10:48.646307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:10:48.646307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:10:48.646307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:10:48.646307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:48.646307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:48.646307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:48.646307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:48.646307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:48.646307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:48.646307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:48.646307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:48.646307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:48.646307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:48.646307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:48.646307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:48.647177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:48.647177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:48.647177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:48.647177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:48.647177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:10:48.647177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:10:48.647177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:10:48.647177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:10:48.647177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:10:48.647177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:10:48.647177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:10:48.647177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:48.649483 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 544/1077] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:10:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:48.652442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d061d37-da6e-4a80-801d-8352377c5185 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:48.680575 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1e1e3-5ef0-4088-b0f7-76f42439de97 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:48.712277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:48.763593 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d061d37-da6e-4a80-801d-8352377c5185 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:48.790266 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1e1e3-5ef0-4088-b0f7-76f42439de97 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:48.803238 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9297961-8999-43e3-8bcf-820195bbe2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:48.803856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9297961-8999-43e3-8bcf-820195bbe2ea None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:48.804133 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9297961-8999-43e3-8bcf-820195bbe2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:48.804410 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9297961-8999-43e3-8bcf-820195bbe2ea None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:48.805061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9297961-8999-43e3-8bcf-820195bbe2ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:48.805390 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9297961-8999-43e3-8bcf-820195bbe2ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:48.806748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9297961-8999-43e3-8bcf-820195bbe2ea None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:48.816240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 534/1078] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:10:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:48.821789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9297961-8999-43e3-8bcf-820195bbe2ea None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:48.833635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dadb5-ce95-4a85-824f-ecb9a91a92ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:48.834092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dadb5-ce95-4a85-824f-ecb9a91a92ce None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:48.834369 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dadb5-ce95-4a85-824f-ecb9a91a92ce None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:48.834639 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dadb5-ce95-4a85-824f-ecb9a91a92ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:48.869345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:48.908035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12dadb5-ce95-4a85-824f-ecb9a91a92ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:48.909435 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12dadb5-ce95-4a85-824f-ecb9a91a92ce None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:48.928469 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9297961-8999-43e3-8bcf-820195bbe2ea None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:48.938185 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:48.967651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e875c3e-37b7-4a69-ac7e-a3a18d63ab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:48.968282 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e875c3e-37b7-4a69-ac7e-a3a18d63ab71 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:48.968559 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e875c3e-37b7-4a69-ac7e-a3a18d63ab71 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:48.968830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e875c3e-37b7-4a69-ac7e-a3a18d63ab71 None admin] PATH_INFO: `/v3/users/dc6626d1a6bd42ef82f015a89d171065` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:48.970015 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e875c3e-37b7-4a69-ac7e-a3a18d63ab71 None admin] RBAC: Authorizing `identity:get_user(user_id=dc6626d1a6bd42ef82f015a89d171065)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:48.971428 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e875c3e-37b7-4a69-ac7e-a3a18d63ab71 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:48.972985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 535/1079] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:10:48 2022] GET /identity/v3/users/dc6626d1a6bd42ef82f015a89d171065 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:48.978559 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e875c3e-37b7-4a69-ac7e-a3a18d63ab71 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:49.021056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 545/1080] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:10:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 12:10:49.026838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dadb5-ce95-4a85-824f-ecb9a91a92ce None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:49.088765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e875c3e-37b7-4a69-ac7e-a3a18d63ab71 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:49.106180 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:49.129832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc77de7-e6ca-4283-abb7-fa46ca1f38c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:49.130467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc77de7-e6ca-4283-abb7-fa46ca1f38c1 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:49.130865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc77de7-e6ca-4283-abb7-fa46ca1f38c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:49.131146 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc77de7-e6ca-4283-abb7-fa46ca1f38c1 None admin] PATH_INFO: `/v3/projects/a2cea9c48e8e4299aedfbccd5e4c29db` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:49.131447 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dadb5-ce95-4a85-824f-ecb9a91a92ce None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:49.132287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc77de7-e6ca-4283-abb7-fa46ca1f38c1 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cea9c48e8e4299aedfbccd5e4c29db)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:49.133745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc77de7-e6ca-4283-abb7-fa46ca1f38c1 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:49.135525 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 536/1081] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:10:48 2022] GET /identity/v3/projects/a2cea9c48e8e4299aedfbccd5e4c29db => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:49.141923 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc77de7-e6ca-4283-abb7-fa46ca1f38c1 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:49.159717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:49.172746 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336b105-f344-4d8b-92e6-795af4009c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:49.173442 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336b105-f344-4d8b-92e6-795af4009c37 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:49.173685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336b105-f344-4d8b-92e6-795af4009c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:49.173950 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336b105-f344-4d8b-92e6-795af4009c37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:49.186418 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336b105-f344-4d8b-92e6-795af4009c37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:49.187584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336b105-f344-4d8b-92e6-795af4009c37 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:49.203420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-5336b105-f344-4d8b-92e6-795af4009c37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:49.203420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:49.203420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:49.203420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:49.203420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:49.203420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:49.203420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:49.203420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:49.203420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:49.203420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:49.203420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:49.203420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:49.203420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:49.203420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:49.203420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:49.203420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:49.204405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:49.204405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:49.204405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:49.204405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:49.204405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:49.204405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:49.204405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:49.204405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:49.204405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:49.204405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:49.204405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:49.204405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:49.204405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:49.204405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:49.204405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:49.204405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:49.204405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:49.205359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:49.205359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:49.205359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:49.205359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 546/1082] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:49.210558 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336b105-f344-4d8b-92e6-795af4009c37 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:49.258242 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc77de7-e6ca-4283-abb7-fa46ca1f38c1 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:49.292063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ba735-918f-491e-8d9f-ac22c005d23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:49.292715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ba735-918f-491e-8d9f-ac22c005d23e None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:49.292992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ba735-918f-491e-8d9f-ac22c005d23e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:49.293295 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ba735-918f-491e-8d9f-ac22c005d23e None admin] PATH_INFO: `/v3/projects/a2cea9c48e8e4299aedfbccd5e4c29db/users/dc6626d1a6bd42ef82f015a89d171065/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:49.295695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ba735-918f-491e-8d9f-ac22c005d23e None admin] RBAC: Authorizing `identity:create_grant(project_id=a2cea9c48e8e4299aedfbccd5e4c29db, user_id=dc6626d1a6bd42ef82f015a89d171065, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:49.297123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ba735-918f-491e-8d9f-ac22c005d23e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:49.321159 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 537/1083] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:10:49 2022] PUT /identity/v3/projects/a2cea9c48e8e4299aedfbccd5e4c29db/users/dc6626d1a6bd42ef82f015a89d171065/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:10:49.329813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336b105-f344-4d8b-92e6-795af4009c37 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:49.367363 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ce48f-9e99-4768-b9e6-f4d396190a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:49.367990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ce48f-9e99-4768-b9e6-f4d396190a7f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:49.368268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ce48f-9e99-4768-b9e6-f4d396190a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:49.368540 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ce48f-9e99-4768-b9e6-f4d396190a7f None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:49.369218 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ce48f-9e99-4768-b9e6-f4d396190a7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:49.369521 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ce48f-9e99-4768-b9e6-f4d396190a7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:49.370916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ce48f-9e99-4768-b9e6-f4d396190a7f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:49.384132 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 547/1084] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:50.077022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d74ad2-88a8-4ddd-a032-409ed34da775 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:50.077609 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d74ad2-88a8-4ddd-a032-409ed34da775 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:50.077609 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d74ad2-88a8-4ddd-a032-409ed34da775 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:50.078922 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 538/1085] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:50.084251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adc727-1229-443e-8811-b9732c10447a None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:50.084657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adc727-1229-443e-8811-b9732c10447a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:50.084793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adc727-1229-443e-8811-b9732c10447a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:50.103191 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-46adc727-1229-443e-8811-b9732c10447a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:50.104366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-46adc727-1229-443e-8811-b9732c10447a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:50.137317 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:50.170800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 548/1086] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:50.436298 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d74ad2-88a8-4ddd-a032-409ed34da775 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:50.478483 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:50.537509 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d74ad2-88a8-4ddd-a032-409ed34da775 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:50.571427 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3084f-31d6-4124-8f3b-5fcab3747e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:50.572045 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3084f-31d6-4124-8f3b-5fcab3747e70 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:50.572321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3084f-31d6-4124-8f3b-5fcab3747e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:50.572590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3084f-31d6-4124-8f3b-5fcab3747e70 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:50.585452 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3084f-31d6-4124-8f3b-5fcab3747e70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:10:50.591022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-3ff3084f-31d6-4124-8f3b-5fcab3747e70 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 12:10:50.591022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:50.591022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:50.591022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:50.591022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:50.591022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:50.591022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:50.591022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:50.591022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:50.591022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:50.591022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:50.591022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:50.591022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:10:50.591022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:10:50.591022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:10:50.591022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:10:50.592161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:10:50.592161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:10:50.592161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:10:50.592161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:10:50.592161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:50.592161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:50.592161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:50.592161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:50.592161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:50.592161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:50.592161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:50.592161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:50.592161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:50.592161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:50.592161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:50.592161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:50.593542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:50.593542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:50.593542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:50.593542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:50.593542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:10:50.593542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:10:50.593542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:10:50.593542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:10:50.593542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:10:50.593542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:10:50.593542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 12:10:50.593542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:50.593542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 539/1087] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Sep 28 12:10:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:50.598161 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46adc727-1229-443e-8811-b9732c10447a None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:50.694453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46adc727-1229-443e-8811-b9732c10447a None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:50.726694 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7b9b2-5923-40b8-9f24-e190c7b53e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:50.727295 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7b9b2-5923-40b8-9f24-e190c7b53e84 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:50.727574 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7b9b2-5923-40b8-9f24-e190c7b53e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:50.727845 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7b9b2-5923-40b8-9f24-e190c7b53e84 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:50.728492 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7b9b2-5923-40b8-9f24-e190c7b53e84 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:50.728792 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7b9b2-5923-40b8-9f24-e190c7b53e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:50.730214 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc7b9b2-5923-40b8-9f24-e190c7b53e84 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:50.738871 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 549/1088] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Sep 28 12:10:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:50.744668 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3084f-31d6-4124-8f3b-5fcab3747e70 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:50.838498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:50.841302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3084f-31d6-4124-8f3b-5fcab3747e70 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:50.875469 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe99af-9017-4580-97ac-3d6853569a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:50.876088 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe99af-9017-4580-97ac-3d6853569a28 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:50.876368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe99af-9017-4580-97ac-3d6853569a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:50.876639 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe99af-9017-4580-97ac-3d6853569a28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:50.887859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe99af-9017-4580-97ac-3d6853569a28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:50.889365 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe99af-9017-4580-97ac-3d6853569a28 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:50.903082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-7efe99af-9017-4580-97ac-3d6853569a28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:50.903082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:50.903082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:50.903082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:50.903082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:50.903082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:50.903082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:50.903082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:50.903082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:50.903082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:50.903082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:50.903082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:50.903082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:50.903082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:50.903082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:50.903082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:50.904105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:50.904105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:50.904105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:50.904105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:50.904105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:50.904105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:50.904105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:50.904105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:50.904105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:50.904105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:50.904105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:50.904105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:50.904105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:50.904105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:50.904105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:50.904105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:50.904105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:50.905658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:50.905658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:50.905658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:50.905658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 540/1089] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:50.910914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7b9b2-5923-40b8-9f24-e190c7b53e84 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:50.942061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:51.008677 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7b9b2-5923-40b8-9f24-e190c7b53e84 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:51.041147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcc29d-e2ac-492d-9635-1525807d603a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:51.041754 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcc29d-e2ac-492d-9635-1525807d603a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:51.042034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcc29d-e2ac-492d-9635-1525807d603a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:51.042308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcc29d-e2ac-492d-9635-1525807d603a None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:51.042937 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcc29d-e2ac-492d-9635-1525807d603a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:51.043239 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcc29d-e2ac-492d-9635-1525807d603a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:51.044587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcc29d-e2ac-492d-9635-1525807d603a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:51.056544 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 550/1090] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:51.062860 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe99af-9017-4580-97ac-3d6853569a28 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:51.163907 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe99af-9017-4580-97ac-3d6853569a28 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:51.199496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b44fc-bddb-4917-b16b-cc83f8bc7bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:51.200126 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b44fc-bddb-4917-b16b-cc83f8bc7bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:51.200410 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b44fc-bddb-4917-b16b-cc83f8bc7bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:51.200687 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b44fc-bddb-4917-b16b-cc83f8bc7bd1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:51.209039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b44fc-bddb-4917-b16b-cc83f8bc7bd1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:51.210506 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b44fc-bddb-4917-b16b-cc83f8bc7bd1 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:51.221969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-561b44fc-bddb-4917-b16b-cc83f8bc7bd1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:10:51.221969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:51.221969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:51.221969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:51.221969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:51.221969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:51.221969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:51.221969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:51.221969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:51.221969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:51.221969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:51.221969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:51.221969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:10:51.221969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:10:51.221969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:10:51.221969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:51.223111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:51.228979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:51.228979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:51.228979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:10:51.228979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:10:51.228979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:10:51.228979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:10:51.228979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:10:51.228979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:10:51.228979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 12:10:51.228979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:51.228979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 541/1091] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Sep 28 12:10:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:51.230198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcc29d-e2ac-492d-9635-1525807d603a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:51.323684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:51.333522 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcc29d-e2ac-492d-9635-1525807d603a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:51.367988 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f21df-f9ca-4fb9-a3b7-f5d0500e25e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:51.368589 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f21df-f9ca-4fb9-a3b7-f5d0500e25e8 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:51.368870 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f21df-f9ca-4fb9-a3b7-f5d0500e25e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:51.369158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f21df-f9ca-4fb9-a3b7-f5d0500e25e8 None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:51.369896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f21df-f9ca-4fb9-a3b7-f5d0500e25e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:51.370205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f21df-f9ca-4fb9-a3b7-f5d0500e25e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:51.371641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f21df-f9ca-4fb9-a3b7-f5d0500e25e8 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:51.392137 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 551/1092] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Sep 28 12:10:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:51.398175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b44fc-bddb-4917-b16b-cc83f8bc7bd1 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:51.414248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:51.498046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b44fc-bddb-4917-b16b-cc83f8bc7bd1 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:51.499808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6abc1-eea3-4970-8414-d685e65ff37c None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:51.500098 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6abc1-eea3-4970-8414-d685e65ff37c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:51.500365 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6abc1-eea3-4970-8414-d685e65ff37c None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:51.501714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 552/1093] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:10:51.506786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8ec00-f706-4344-81a5-cb30b72e06cf None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:51.507061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8ec00-f706-4344-81a5-cb30b72e06cf None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:51.507328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8ec00-f706-4344-81a5-cb30b72e06cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:51.525842 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-3ea8ec00-f706-4344-81a5-cb30b72e06cf None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:51.527031 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-3ea8ec00-f706-4344-81a5-cb30b72e06cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:51.532981 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd246e-e96b-45a2-9684-8997eec0bca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:51.533609 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd246e-e96b-45a2-9684-8997eec0bca4 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:51.533887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd246e-e96b-45a2-9684-8997eec0bca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:51.534157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbd246e-e96b-45a2-9684-8997eec0bca4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:51.593799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 553/1094] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:10:51.603641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd246e-e96b-45a2-9684-8997eec0bca4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:51.605082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbd246e-e96b-45a2-9684-8997eec0bca4 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:51.619197 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:51.706596 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 542/1095] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:10:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:10:51.707734 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8ec00-f706-4344-81a5-cb30b72e06cf None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:51.785192 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:51.806124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8ec00-f706-4344-81a5-cb30b72e06cf None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:51.838234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157300d-da42-42cc-9d39-4e97c5ecd36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:51.838868 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157300d-da42-42cc-9d39-4e97c5ecd36a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:51.839127 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157300d-da42-42cc-9d39-4e97c5ecd36a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:51.839399 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157300d-da42-42cc-9d39-4e97c5ecd36a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:51.849814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157300d-da42-42cc-9d39-4e97c5ecd36a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:51.851186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157300d-da42-42cc-9d39-4e97c5ecd36a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:51.862461 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd246e-e96b-45a2-9684-8997eec0bca4 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:51.866527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-e157300d-da42-42cc-9d39-4e97c5ecd36a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:51.866527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:51.866527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:51.866527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:51.866527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:51.866527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:51.866527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:51.866527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:51.866527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:51.866527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:51.866527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:51.866527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:51.866527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:10:51.866527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:10:51.866527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:10:51.866527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:10:51.867443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:51.867443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:51.867443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:51.867443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:51.867443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:51.867443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:51.867443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:51.867443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:51.867443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:51.867443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:51.867443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:51.867443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:51.867443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:51.867443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:51.867443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:51.867443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:51.867443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:10:51.873025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:10:51.873025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:10:51.873025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:51.873025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 554/1096] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:51.873778 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157300d-da42-42cc-9d39-4e97c5ecd36a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:51.961827 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbd246e-e96b-45a2-9684-8997eec0bca4 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:51.970286 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157300d-da42-42cc-9d39-4e97c5ecd36a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:51.975820 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:51.997668 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2459f2-cb21-4273-976e-3f5a6fd1aa6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:51.998299 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2459f2-cb21-4273-976e-3f5a6fd1aa6e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:51.998589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2459f2-cb21-4273-976e-3f5a6fd1aa6e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:51.998860 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2459f2-cb21-4273-976e-3f5a6fd1aa6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:51.999662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:52.004860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328aa45-6fc9-4a71-9921-6bf61fd6f476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:52.005503 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328aa45-6fc9-4a71-9921-6bf61fd6f476 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:52.005778 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328aa45-6fc9-4a71-9921-6bf61fd6f476 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:52.006111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328aa45-6fc9-4a71-9921-6bf61fd6f476 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:52.006741 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328aa45-6fc9-4a71-9921-6bf61fd6f476 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:52.007037 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328aa45-6fc9-4a71-9921-6bf61fd6f476 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:52.008383 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328aa45-6fc9-4a71-9921-6bf61fd6f476 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:52.008974 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2459f2-cb21-4273-976e-3f5a6fd1aa6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:10:52.013859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-da2459f2-cb21-4273-976e-3f5a6fd1aa6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:10:52.013859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:52.013859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:52.013859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:52.013859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:52.013859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:52.013859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:52.013859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:52.013859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:52.013859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:52.013859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:52.013859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:52.013859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:10:52.013859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:10:52.013859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:10:52.013859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:10:52.015006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:10:52.015006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:10:52.015006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:10:52.015006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:10:52.015006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:52.015006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:52.015006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:52.015006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:52.015006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:52.015006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:52.015006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:52.015006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:52.015006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:52.015006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:52.015006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:52.015006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:52.015872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:52.015872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:52.015872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:52.015872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:52.015872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:10:52.015872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:10:52.015872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:10:52.015872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:10:52.015872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:10:52.015872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:10:52.015872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:10:52.015872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:10:52.015872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 543/1097] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:10:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:10:52.021009 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2459f2-cb21-4273-976e-3f5a6fd1aa6e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:52.026200 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 555/1098] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:52.027160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328aa45-6fc9-4a71-9921-6bf61fd6f476 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:52.121006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2459f2-cb21-4273-976e-3f5a6fd1aa6e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:52.124797 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328aa45-6fc9-4a71-9921-6bf61fd6f476 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:52.155597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ab804-4530-425e-868f-a843ce61deb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:52.156190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ab804-4530-425e-868f-a843ce61deb0 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:52.156468 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ab804-4530-425e-868f-a843ce61deb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:52.156736 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ab804-4530-425e-868f-a843ce61deb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:52.157418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ab804-4530-425e-868f-a843ce61deb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:52.157718 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ab804-4530-425e-868f-a843ce61deb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:52.158932 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60169557-0747-4864-bbae-f8625b2505a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:52.159079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ab804-4530-425e-868f-a843ce61deb0 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:52.159554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60169557-0747-4864-bbae-f8625b2505a1 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:52.159832 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60169557-0747-4864-bbae-f8625b2505a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:52.160106 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60169557-0747-4864-bbae-f8625b2505a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:52.166731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60169557-0747-4864-bbae-f8625b2505a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:52.168164 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60169557-0747-4864-bbae-f8625b2505a1 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:52.172364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 544/1099] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:10:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:52.173305 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ab804-4530-425e-868f-a843ce61deb0 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:52.181143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-60169557-0747-4864-bbae-f8625b2505a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:52.181143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:10:52.181143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:10:52.181143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:10:52.181143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:10:52.181143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:10:52.181143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:10:52.181143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:10:52.181143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:10:52.181143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:10:52.181143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:10:52.181143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:10:52.181143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:10:52.181143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:10:52.181143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:10:52.182101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:10:52.182101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:10:52.182101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:10:52.182101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:10:52.182101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:10:52.182101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:10:52.182101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:10:52.182101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:10:52.182101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:10:52.182101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:10:52.182101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:10:52.182101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:10:52.182101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:10:52.182101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:10:52.182101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:10:52.182101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:10:52.182101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:10:52.183056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:10:52.183056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:10:52.183056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:10:52.183056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:10:52.183056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:10:52.183056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:10:52.183056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:10:52.183056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:10:52.187102 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 556/1100] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:10:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:10:52.187925 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60169557-0747-4864-bbae-f8625b2505a1 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:52.207439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:52.241193 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:52.275097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ab804-4530-425e-868f-a843ce61deb0 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:52.289537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60169557-0747-4864-bbae-f8625b2505a1 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:52.308768 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5867d97-4d14-48e3-a74e-87ee4d777c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:52.309426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5867d97-4d14-48e3-a74e-87ee4d777c63 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:52.309711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5867d97-4d14-48e3-a74e-87ee4d777c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:52.309985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5867d97-4d14-48e3-a74e-87ee4d777c63 None admin] PATH_INFO: `/v3/users/dc6626d1a6bd42ef82f015a89d171065` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:52.311193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5867d97-4d14-48e3-a74e-87ee4d777c63 None admin] RBAC: Authorizing `identity:get_user(user_id=dc6626d1a6bd42ef82f015a89d171065)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:52.312687 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5867d97-4d14-48e3-a74e-87ee4d777c63 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:52.314297 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 545/1101] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:10:52 2022] GET /identity/v3/users/dc6626d1a6bd42ef82f015a89d171065 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:52.319986 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5867d97-4d14-48e3-a74e-87ee4d777c63 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:52.325100 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac9251-106e-42aa-8a71-c0c5b6b0c4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:52.325737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac9251-106e-42aa-8a71-c0c5b6b0c4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:52.326016 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac9251-106e-42aa-8a71-c0c5b6b0c4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:52.326824 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac9251-106e-42aa-8a71-c0c5b6b0c4d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:52.327516 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac9251-106e-42aa-8a71-c0c5b6b0c4d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:52.327824 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac9251-106e-42aa-8a71-c0c5b6b0c4d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:52.329339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac9251-106e-42aa-8a71-c0c5b6b0c4d1 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:52.341576 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 557/1102] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:10:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:52.348038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac9251-106e-42aa-8a71-c0c5b6b0c4d1 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:52.419992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5867d97-4d14-48e3-a74e-87ee4d777c63 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:52.421559 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:52.445737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac9251-106e-42aa-8a71-c0c5b6b0c4d1 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:52.455370 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a1518-6556-4e36-a382-a1f1a6b9f347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:52.455976 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a1518-6556-4e36-a382-a1f1a6b9f347 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:52.456269 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a1518-6556-4e36-a382-a1f1a6b9f347 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:52.456542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a1518-6556-4e36-a382-a1f1a6b9f347 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:52.458735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:52.479957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ed733-e7cd-49d9-bf5a-c84c14ed5190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:52.480549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ed733-e7cd-49d9-bf5a-c84c14ed5190 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:52.480827 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ed733-e7cd-49d9-bf5a-c84c14ed5190 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:52.481243 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ed733-e7cd-49d9-bf5a-c84c14ed5190 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:52.482462 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ed733-e7cd-49d9-bf5a-c84c14ed5190 None admin] RBAC: Adding query filter params (role.id=8af05c40a1604332a08a2c6ba7ad6de7, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=982b82e8ac424318a934f9149fb91706) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:52.483541 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ed733-e7cd-49d9-bf5a-c84c14ed5190 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:52.486466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ed733-e7cd-49d9-bf5a-c84c14ed5190 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:52.492666 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 558/1103] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:10:52 2022] GET /identity/v3/role_assignments?user.id=982b82e8ac424318a934f9149fb91706&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=8af05c40a1604332a08a2c6ba7ad6de7 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:10:52.526877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a1518-6556-4e36-a382-a1f1a6b9f347 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:52.528440 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a1518-6556-4e36-a382-a1f1a6b9f347 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:52.634468 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 546/1104] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:10:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 12:10:52.640628 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ed733-e7cd-49d9-bf5a-c84c14ed5190 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:52.742551 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ed733-e7cd-49d9-bf5a-c84c14ed5190 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:52.776485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b7395-2dc5-400f-a450-fc674ad6c1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:52.777110 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b7395-2dc5-400f-a450-fc674ad6c1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:52.777427 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b7395-2dc5-400f-a450-fc674ad6c1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:52.777700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b7395-2dc5-400f-a450-fc674ad6c1e7 None admin] PATH_INFO: `/v3/projects/a2cea9c48e8e4299aedfbccd5e4c29db` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:52.778798 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b7395-2dc5-400f-a450-fc674ad6c1e7 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cea9c48e8e4299aedfbccd5e4c29db)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:52.780859 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b7395-2dc5-400f-a450-fc674ad6c1e7 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:52.782542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 559/1105] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:10:52 2022] GET /identity/v3/projects/a2cea9c48e8e4299aedfbccd5e4c29db => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:52.788658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a1518-6556-4e36-a382-a1f1a6b9f347 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:52.790835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:52.892835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a1518-6556-4e36-a382-a1f1a6b9f347 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:52.927956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab964673-da60-45b7-81dc-efabc1dc1b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:52.928571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab964673-da60-45b7-81dc-efabc1dc1b11 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:52.928842 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab964673-da60-45b7-81dc-efabc1dc1b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:52.929115 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab964673-da60-45b7-81dc-efabc1dc1b11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:52.929889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab964673-da60-45b7-81dc-efabc1dc1b11 None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=a2cea9c48e8e4299aedfbccd5e4c29db, user.id=dc6626d1a6bd42ef82f015a89d171065) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:52.932061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab964673-da60-45b7-81dc-efabc1dc1b11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:52.932061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab964673-da60-45b7-81dc-efabc1dc1b11 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:52.937561 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 547/1106] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:10:52 2022] GET /identity/v3/role_assignments?user.id=dc6626d1a6bd42ef82f015a89d171065&scope.project.id=a2cea9c48e8e4299aedfbccd5e4c29db&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:55.104122 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e52327-7b0a-431f-ba56-b46f4802ee55 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:55.104678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e52327-7b0a-431f-ba56-b46f4802ee55 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:55.104678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e52327-7b0a-431f-ba56-b46f4802ee55 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:55.106024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 560/1107] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:55.112322 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2293d-0988-4e59-9844-552a6e2f3af1 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:55.112709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2293d-0988-4e59-9844-552a6e2f3af1 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:55.112874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2293d-0988-4e59-9844-552a6e2f3af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:55.131212 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-5fa2293d-0988-4e59-9844-552a6e2f3af1 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:55.132391 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5fa2293d-0988-4e59-9844-552a6e2f3af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:55.191092 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:55.198628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 548/1108] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:55.467653 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e52327-7b0a-431f-ba56-b46f4802ee55 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:55.513147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:55.568517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e52327-7b0a-431f-ba56-b46f4802ee55 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:55.600946 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff23772-217f-4191-b347-44ac44430507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:55.601610 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff23772-217f-4191-b347-44ac44430507 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:55.601887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff23772-217f-4191-b347-44ac44430507 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:55.602156 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff23772-217f-4191-b347-44ac44430507 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:55.603258 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff23772-217f-4191-b347-44ac44430507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:55.604689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff23772-217f-4191-b347-44ac44430507 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:55.606284 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 561/1109] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:10:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:55.612601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2293d-0988-4e59-9844-552a6e2f3af1 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:55.712130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2293d-0988-4e59-9844-552a6e2f3af1 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:55.746612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f64459-1820-4ad1-b6c9-435810f55b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:55.747123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f64459-1820-4ad1-b6c9-435810f55b11 None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:55.747400 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f64459-1820-4ad1-b6c9-435810f55b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:55.747674 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f64459-1820-4ad1-b6c9-435810f55b11 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:55.748865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f64459-1820-4ad1-b6c9-435810f55b11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:55.750396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f64459-1820-4ad1-b6c9-435810f55b11 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:55.781207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 549/1110] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:10:55 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:10:57.956021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994fe14-c493-47c5-a18d-bb135f030141 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:57.956595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994fe14-c493-47c5-a18d-bb135f030141 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:57.956595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994fe14-c493-47c5-a18d-bb135f030141 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:57.957933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 562/1111] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:10:57.963085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617617f6-da09-4c48-8648-f055e5b6ea3c None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:57.963464 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617617f6-da09-4c48-8648-f055e5b6ea3c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:57.963631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617617f6-da09-4c48-8648-f055e5b6ea3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:57.981902 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-617617f6-da09-4c48-8648-f055e5b6ea3c None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:10:57.983120 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-617617f6-da09-4c48-8648-f055e5b6ea3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:10:58.029230 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:58.049557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 550/1112] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:10:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:10:58.319796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994fe14-c493-47c5-a18d-bb135f030141 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:58.421935 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994fe14-c493-47c5-a18d-bb135f030141 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:58.434145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:58.456079 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60528805-4499-4c21-bec1-e27056e396df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:58.456697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60528805-4499-4c21-bec1-e27056e396df None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:58.456973 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60528805-4499-4c21-bec1-e27056e396df None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:58.457272 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60528805-4499-4c21-bec1-e27056e396df None admin] PATH_INFO: `/v3/roles/8e9576d16f194938b86a1d1e087895bd` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:58.458645 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60528805-4499-4c21-bec1-e27056e396df None admin] RBAC: Authorizing `identity:get_role(role_id=8e9576d16f194938b86a1d1e087895bd)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:58.460004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60528805-4499-4c21-bec1-e27056e396df None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:58.461335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 563/1113] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:10:58 2022] GET /identity/v3/roles/8e9576d16f194938b86a1d1e087895bd => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:58.467094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617617f6-da09-4c48-8648-f055e5b6ea3c None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:58.565218 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617617f6-da09-4c48-8648-f055e5b6ea3c None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:58.598864 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d07004-f5d8-4d92-870d-c20227530859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:58.599421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d07004-f5d8-4d92-870d-c20227530859 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:58.599701 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d07004-f5d8-4d92-870d-c20227530859 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:58.599973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d07004-f5d8-4d92-870d-c20227530859 None admin] PATH_INFO: `/v3/users/ea570a739e884ff5a19a07e5b672fe1d` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:58.613124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d07004-f5d8-4d92-870d-c20227530859 None admin] RBAC: Authorizing `identity:get_user(user_id=ea570a739e884ff5a19a07e5b672fe1d)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:58.614590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d07004-f5d8-4d92-870d-c20227530859 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:58.616191 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 551/1114] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:10:58 2022] GET /identity/v3/users/ea570a739e884ff5a19a07e5b672fe1d => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:58.621915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60528805-4499-4c21-bec1-e27056e396df None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:58.718433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60528805-4499-4c21-bec1-e27056e396df None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:58.751217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b228ca-8d1a-4575-ac51-1dc1ca99d7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:58.751809 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b228ca-8d1a-4575-ac51-1dc1ca99d7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:58.752090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b228ca-8d1a-4575-ac51-1dc1ca99d7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:58.752360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b228ca-8d1a-4575-ac51-1dc1ca99d7c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:58.798999 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:58.817553 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b228ca-8d1a-4575-ac51-1dc1ca99d7c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:58.818982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b228ca-8d1a-4575-ac51-1dc1ca99d7c3 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:58.914529 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 564/1115] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:10:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:10:58.921333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d07004-f5d8-4d92-870d-c20227530859 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:58.964086 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:59.022774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d07004-f5d8-4d92-870d-c20227530859 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:59.056461 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf47f7-c36d-48d2-9887-f61d2be759d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:59.057084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf47f7-c36d-48d2-9887-f61d2be759d4 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:59.057395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf47f7-c36d-48d2-9887-f61d2be759d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:59.057671 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf47f7-c36d-48d2-9887-f61d2be759d4 None admin] PATH_INFO: `/v3/projects/a2cea9c48e8e4299aedfbccd5e4c29db` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:59.058781 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf47f7-c36d-48d2-9887-f61d2be759d4 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cea9c48e8e4299aedfbccd5e4c29db)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:59.060201 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf47f7-c36d-48d2-9887-f61d2be759d4 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:59.062798 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 552/1116] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:10:58 2022] GET /identity/v3/projects/a2cea9c48e8e4299aedfbccd5e4c29db => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:10:59.068883 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b228ca-8d1a-4575-ac51-1dc1ca99d7c3 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:10:59.145263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:10:59.164742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b228ca-8d1a-4575-ac51-1dc1ca99d7c3 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:10:59.197678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee689fbb-ea4d-4ea5-a237-418c9a0fff25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:10:59.198273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee689fbb-ea4d-4ea5-a237-418c9a0fff25 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:10:59.198552 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee689fbb-ea4d-4ea5-a237-418c9a0fff25 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:10:59.198821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee689fbb-ea4d-4ea5-a237-418c9a0fff25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:10:59.199529 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee689fbb-ea4d-4ea5-a237-418c9a0fff25 None admin] RBAC: Adding query filter params (role.id=8e9576d16f194938b86a1d1e087895bd, scope.project.id=a2cea9c48e8e4299aedfbccd5e4c29db, user.id=ea570a739e884ff5a19a07e5b672fe1d) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:10:59.199835 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee689fbb-ea4d-4ea5-a237-418c9a0fff25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:10:59.201296 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee689fbb-ea4d-4ea5-a237-418c9a0fff25 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:10:59.206420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 565/1117] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:10:59 2022] GET /identity/v3/role_assignments?user.id=ea570a739e884ff5a19a07e5b672fe1d&scope.project.id=a2cea9c48e8e4299aedfbccd5e4c29db&role.id=8e9576d16f194938b86a1d1e087895bd => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:01.332326 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aec849-0050-426f-ad79-9b85bb9be617 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:01.332873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aec849-0050-426f-ad79-9b85bb9be617 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:01.332873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aec849-0050-426f-ad79-9b85bb9be617 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:01.334281 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 553/1118] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:11:01.339325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a10037-2ccb-4c6c-942a-d60b4c9ed602 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:01.339755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a10037-2ccb-4c6c-942a-d60b4c9ed602 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:01.339875 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a10037-2ccb-4c6c-942a-d60b4c9ed602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:01.358561 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-62a10037-2ccb-4c6c-942a-d60b4c9ed602 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:11:01.359729 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-62a10037-2ccb-4c6c-942a-d60b4c9ed602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:11:01.424264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 566/1119] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:11:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:11:01.697574 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aec849-0050-426f-ad79-9b85bb9be617 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:01.798984 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aec849-0050-426f-ad79-9b85bb9be617 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:01.807999 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:01.833634 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fed6d-1622-46a4-b450-0db171eee7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:01.834259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fed6d-1622-46a4-b450-0db171eee7ed None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:01.834548 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fed6d-1622-46a4-b450-0db171eee7ed None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:01.834828 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fed6d-1622-46a4-b450-0db171eee7ed None admin] PATH_INFO: `/v3/roles/8e9576d16f194938b86a1d1e087895bd` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:01.836213 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fed6d-1622-46a4-b450-0db171eee7ed None admin] RBAC: Authorizing `identity:get_role(role_id=8e9576d16f194938b86a1d1e087895bd)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:01.837615 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fed6d-1622-46a4-b450-0db171eee7ed None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:01.838913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 554/1120] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:11:01 2022] GET /identity/v3/roles/8e9576d16f194938b86a1d1e087895bd => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:01.845453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a10037-2ccb-4c6c-942a-d60b4c9ed602 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:01.916847 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:01.940974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a10037-2ccb-4c6c-942a-d60b4c9ed602 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:01.974413 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d2563-069c-4227-979a-44142e2ab83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:01.974980 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d2563-069c-4227-979a-44142e2ab83e None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:01.975262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d2563-069c-4227-979a-44142e2ab83e None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:01.975537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d2563-069c-4227-979a-44142e2ab83e None admin] PATH_INFO: `/v3/users/ea570a739e884ff5a19a07e5b672fe1d` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:01.976641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d2563-069c-4227-979a-44142e2ab83e None admin] RBAC: Authorizing `identity:get_user(user_id=ea570a739e884ff5a19a07e5b672fe1d)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:01.978091 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d2563-069c-4227-979a-44142e2ab83e None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:01.979648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 567/1121] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:11:01 2022] GET /identity/v3/users/ea570a739e884ff5a19a07e5b672fe1d => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:01.985989 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fed6d-1622-46a4-b450-0db171eee7ed None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:02.083150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fed6d-1622-46a4-b450-0db171eee7ed None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:02.116526 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22323ba-fb5d-4f40-a83b-e598b5ca714f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:02.117162 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22323ba-fb5d-4f40-a83b-e598b5ca714f None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:02.117453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22323ba-fb5d-4f40-a83b-e598b5ca714f None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:02.117730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22323ba-fb5d-4f40-a83b-e598b5ca714f None admin] PATH_INFO: `/v3/projects/a2cea9c48e8e4299aedfbccd5e4c29db` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:02.118834 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22323ba-fb5d-4f40-a83b-e598b5ca714f None admin] RBAC: Authorizing `identity:get_project(project_id=a2cea9c48e8e4299aedfbccd5e4c29db)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:02.120254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22323ba-fb5d-4f40-a83b-e598b5ca714f None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:02.121904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 555/1122] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:11:01 2022] GET /identity/v3/projects/a2cea9c48e8e4299aedfbccd5e4c29db => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:02.127833 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d2563-069c-4227-979a-44142e2ab83e None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:02.224272 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d2563-069c-4227-979a-44142e2ab83e None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:02.256838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7f997-5a8c-45fa-8f85-c756cda2258f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:02.257449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7f997-5a8c-45fa-8f85-c756cda2258f None admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:02.257726 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7f997-5a8c-45fa-8f85-c756cda2258f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:02.257994 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7f997-5a8c-45fa-8f85-c756cda2258f None admin] PATH_INFO: `/v3/projects/a2cea9c48e8e4299aedfbccd5e4c29db/users/ea570a739e884ff5a19a07e5b672fe1d/roles/8e9576d16f194938b86a1d1e087895bd` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:02.260067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7f997-5a8c-45fa-8f85-c756cda2258f None admin] RBAC: Authorizing `identity:create_grant(project_id=a2cea9c48e8e4299aedfbccd5e4c29db, user_id=ea570a739e884ff5a19a07e5b672fe1d, role_id=8e9576d16f194938b86a1d1e087895bd)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:02.261496 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7f997-5a8c-45fa-8f85-c756cda2258f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:02.274207 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 568/1123] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:11:02 2022] PUT /identity/v3/projects/a2cea9c48e8e4299aedfbccd5e4c29db/users/ea570a739e884ff5a19a07e5b672fe1d/roles/8e9576d16f194938b86a1d1e087895bd => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:11:04.389429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd1471-e086-4158-804c-ce838362947b None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:04.389983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd1471-e086-4158-804c-ce838362947b None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:04.389983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd1471-e086-4158-804c-ce838362947b None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:04.391314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 556/1124] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:11:04.396471 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fb195-db89-45eb-9418-1ae0204da7f6 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:04.396916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fb195-db89-45eb-9418-1ae0204da7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:04.397043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fb195-db89-45eb-9418-1ae0204da7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:04.403176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:04.416826 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-d69fb195-db89-45eb-9418-1ae0204da7f6 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:11:04.418012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d69fb195-db89-45eb-9418-1ae0204da7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:11:04.481275 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 569/1125] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:11:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:11:04.747274 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd1471-e086-4158-804c-ce838362947b None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:04.793180 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:04.846300 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd1471-e086-4158-804c-ce838362947b None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:04.879444 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60359c70-8e46-475e-baf6-de153af3bd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:04.880057 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60359c70-8e46-475e-baf6-de153af3bd1c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:04.880337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60359c70-8e46-475e-baf6-de153af3bd1c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:04.880606 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60359c70-8e46-475e-baf6-de153af3bd1c None admin] PATH_INFO: `/v3/roles/8e9576d16f194938b86a1d1e087895bd` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:04.881993 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60359c70-8e46-475e-baf6-de153af3bd1c None admin] RBAC: Authorizing `identity:get_role(role_id=8e9576d16f194938b86a1d1e087895bd)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:04.883340 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60359c70-8e46-475e-baf6-de153af3bd1c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:04.884621 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 557/1126] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:11:04 2022] GET /identity/v3/roles/8e9576d16f194938b86a1d1e087895bd => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:04.890378 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fb195-db89-45eb-9418-1ae0204da7f6 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:04.984275 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fb195-db89-45eb-9418-1ae0204da7f6 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:05.012594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:05.017941 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746101b6-ea53-485f-a4d5-b0edacf37589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:05.018565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746101b6-ea53-485f-a4d5-b0edacf37589 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:05.018846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746101b6-ea53-485f-a4d5-b0edacf37589 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:05.019119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746101b6-ea53-485f-a4d5-b0edacf37589 None admin] PATH_INFO: `/v3/users/ea570a739e884ff5a19a07e5b672fe1d` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:05.020223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746101b6-ea53-485f-a4d5-b0edacf37589 None admin] RBAC: Authorizing `identity:get_user(user_id=ea570a739e884ff5a19a07e5b672fe1d)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:05.021676 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746101b6-ea53-485f-a4d5-b0edacf37589 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:05.023235 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 570/1127] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:11:04 2022] GET /identity/v3/users/ea570a739e884ff5a19a07e5b672fe1d => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:05.028918 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60359c70-8e46-475e-baf6-de153af3bd1c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:05.126252 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60359c70-8e46-475e-baf6-de153af3bd1c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:05.138873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:05.159714 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc3cf9-b24e-414e-b01d-29fafccbb984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:05.160308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc3cf9-b24e-414e-b01d-29fafccbb984 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:05.160589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc3cf9-b24e-414e-b01d-29fafccbb984 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:05.160858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc3cf9-b24e-414e-b01d-29fafccbb984 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:05.226069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc3cf9-b24e-414e-b01d-29fafccbb984 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:05.227502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc3cf9-b24e-414e-b01d-29fafccbb984 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:05.322432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 558/1128] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:11:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:11:05.328594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746101b6-ea53-485f-a4d5-b0edacf37589 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:05.423375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746101b6-ea53-485f-a4d5-b0edacf37589 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:05.454983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46b37a-ad65-4c15-9c32-9c8e98f75627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:05.455593 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46b37a-ad65-4c15-9c32-9c8e98f75627 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:05.455870 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46b37a-ad65-4c15-9c32-9c8e98f75627 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:05.456140 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46b37a-ad65-4c15-9c32-9c8e98f75627 None admin] PATH_INFO: `/v3/projects/a2cea9c48e8e4299aedfbccd5e4c29db` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:05.457242 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46b37a-ad65-4c15-9c32-9c8e98f75627 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cea9c48e8e4299aedfbccd5e4c29db)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:05.458644 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46b37a-ad65-4c15-9c32-9c8e98f75627 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:05.460218 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 571/1129] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:11:05 2022] GET /identity/v3/projects/a2cea9c48e8e4299aedfbccd5e4c29db => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:05.466276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc3cf9-b24e-414e-b01d-29fafccbb984 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:05.480585 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:05.564578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc3cf9-b24e-414e-b01d-29fafccbb984 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:05.598740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c361a-874d-4292-a307-b7476d58a34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:05.599274 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c361a-874d-4292-a307-b7476d58a34d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:05.599552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c361a-874d-4292-a307-b7476d58a34d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:05.599823 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c361a-874d-4292-a307-b7476d58a34d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:05.600535 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c361a-874d-4292-a307-b7476d58a34d None admin] RBAC: Adding query filter params (role.id=8e9576d16f194938b86a1d1e087895bd, scope.project.id=a2cea9c48e8e4299aedfbccd5e4c29db, user.id=ea570a739e884ff5a19a07e5b672fe1d) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:11:05.600841 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c361a-874d-4292-a307-b7476d58a34d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:05.602287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c361a-874d-4292-a307-b7476d58a34d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:05.607869 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 559/1130] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:11:05 2022] GET /identity/v3/role_assignments?user.id=ea570a739e884ff5a19a07e5b672fe1d&scope.project.id=a2cea9c48e8e4299aedfbccd5e4c29db&role.id=8e9576d16f194938b86a1d1e087895bd => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:07.756649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eba379-6530-4ba3-8f06-35a37159cc0a None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:07.757206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eba379-6530-4ba3-8f06-35a37159cc0a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:07.757206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eba379-6530-4ba3-8f06-35a37159cc0a None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:07.758533 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 572/1131] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:11:07.764465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee645b-0396-4faa-87e4-62f43872a4a5 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:07.764903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee645b-0396-4faa-87e4-62f43872a4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:07.764996 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee645b-0396-4faa-87e4-62f43872a4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:07.782818 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-aaee645b-0396-4faa-87e4-62f43872a4a5 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:11:07.783981 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-aaee645b-0396-4faa-87e4-62f43872a4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:11:07.849555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 560/1132] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:11:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:11:08.128979 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eba379-6530-4ba3-8f06-35a37159cc0a None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:08.195034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:08.228057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eba379-6530-4ba3-8f06-35a37159cc0a None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:08.260448 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b6796-218f-4bde-b09d-b78f3317a1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:08.261059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b6796-218f-4bde-b09d-b78f3317a1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:08.261369 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b6796-218f-4bde-b09d-b78f3317a1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:08.261639 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b6796-218f-4bde-b09d-b78f3317a1b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:08.262718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b6796-218f-4bde-b09d-b78f3317a1b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:08.264128 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b6796-218f-4bde-b09d-b78f3317a1b0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:08.265704 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 573/1133] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:11:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:08.271475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee645b-0396-4faa-87e4-62f43872a4a5 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:08.279931 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:08.367551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee645b-0396-4faa-87e4-62f43872a4a5 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:08.400292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c183a69-7b1b-4f4b-8222-766ae24b9ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:08.400824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c183a69-7b1b-4f4b-8222-766ae24b9ffa None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:08.401101 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c183a69-7b1b-4f4b-8222-766ae24b9ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:08.401406 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c183a69-7b1b-4f4b-8222-766ae24b9ffa None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:08.402049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c183a69-7b1b-4f4b-8222-766ae24b9ffa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:08.403454 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c183a69-7b1b-4f4b-8222-766ae24b9ffa None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:08.430097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 561/1134] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 12:11:08 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:11:10.586238 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf6463-b7aa-436b-978b-c9010f62799f None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:10.586776 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf6463-b7aa-436b-978b-c9010f62799f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:10.586886 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf6463-b7aa-436b-978b-c9010f62799f None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:10.588120 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 574/1135] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:11:10.593940 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d45f9-8efd-450b-aa8b-2a1a31535f2c None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:10.594364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d45f9-8efd-450b-aa8b-2a1a31535f2c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:10.594494 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d45f9-8efd-450b-aa8b-2a1a31535f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:10.613213 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-0d5d45f9-8efd-450b-aa8b-2a1a31535f2c None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:11:10.614374 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0d5d45f9-8efd-450b-aa8b-2a1a31535f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:11:10.664658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:10.678886 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 562/1136] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:11:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:11:10.944005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf6463-b7aa-436b-978b-c9010f62799f None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:11.042565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf6463-b7aa-436b-978b-c9010f62799f None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:11.074508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479ff4d-b946-4d9d-b3b4-04bc3774a68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:11.075145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479ff4d-b946-4d9d-b3b4-04bc3774a68d None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:11.075424 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479ff4d-b946-4d9d-b3b4-04bc3774a68d None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:11.076237 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479ff4d-b946-4d9d-b3b4-04bc3774a68d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:11.077354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479ff4d-b946-4d9d-b3b4-04bc3774a68d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:11.078773 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479ff4d-b946-4d9d-b3b4-04bc3774a68d None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:11.080295 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 575/1137] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:11:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:11.086024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d45f9-8efd-450b-aa8b-2a1a31535f2c None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:11.181673 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d45f9-8efd-450b-aa8b-2a1a31535f2c None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:11.215527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6e3421-0c0b-43bf-b0e8-497a81044c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:11.216068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e3421-0c0b-43bf-b0e8-497a81044c07 None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:11.216430 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e3421-0c0b-43bf-b0e8-497a81044c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:11.216893 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6e3421-0c0b-43bf-b0e8-497a81044c07 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:11.217711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e3421-0c0b-43bf-b0e8-497a81044c07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:11.218984 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6e3421-0c0b-43bf-b0e8-497a81044c07 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:11.239477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 563/1138] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:11:11 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:11:13.431121 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e11fc-4fcb-4959-92fe-a7ab9690e109 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:13.431646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e11fc-4fcb-4959-92fe-a7ab9690e109 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:13.431646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e11fc-4fcb-4959-92fe-a7ab9690e109 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:13.432991 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 576/1139] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:11:13.438434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90b05d-32df-46af-9110-36d41098c4a0 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:13.438835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90b05d-32df-46af-9110-36d41098c4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:13.438975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90b05d-32df-46af-9110-36d41098c4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:13.456683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-5a90b05d-32df-46af-9110-36d41098c4a0 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:11:13.457874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5a90b05d-32df-46af-9110-36d41098c4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:11:13.496550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:13.523063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 564/1140] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:11:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:11:13.796372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e11fc-4fcb-4959-92fe-a7ab9690e109 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:13.800149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:13.895829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e11fc-4fcb-4959-92fe-a7ab9690e109 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:13.928359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb436d7e-2277-4dfa-a46c-a211af88ca9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:13.928972 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb436d7e-2277-4dfa-a46c-a211af88ca9d None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:13.929283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb436d7e-2277-4dfa-a46c-a211af88ca9d None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:13.929557 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb436d7e-2277-4dfa-a46c-a211af88ca9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:13.939264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb436d7e-2277-4dfa-a46c-a211af88ca9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:11:13.944011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-eb436d7e-2277-4dfa-a46c-a211af88ca9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:11:13.944011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:11:13.944011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:11:13.944011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:11:13.944011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:11:13.944011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:11:13.944011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:11:13.944011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:11:13.944011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:11:13.944011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:11:13.944011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:11:13.944011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:11:13.944011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:11:13.944011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:11:13.944011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:11:13.944011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:11:13.945088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:11:13.945088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:11:13.945088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:11:13.945088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:11:13.945088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:11:13.945088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:11:13.945088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:11:13.945088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:11:13.945088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:11:13.945088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:11:13.945088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:11:13.945088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:11:13.945088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:11:13.945088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:11:13.945088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:11:13.945088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:11:13.946131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:11:13.946131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:11:13.946131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:11:13.946131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:11:13.946131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:11:13.946131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:11:13.946131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:11:13.946131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:11:13.946131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:11:13.946131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:11:13.946131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:11:13.946131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:11:13.946131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 577/1141] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:11:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:11:13.951833 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90b05d-32df-46af-9110-36d41098c4a0 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:14.050490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90b05d-32df-46af-9110-36d41098c4a0 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:14.083463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc662ba-e2e8-4fac-88ff-ab1acbd4fd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:14.084071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc662ba-e2e8-4fac-88ff-ab1acbd4fd8d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:14.084352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc662ba-e2e8-4fac-88ff-ab1acbd4fd8d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:14.084623 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc662ba-e2e8-4fac-88ff-ab1acbd4fd8d None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:14.085310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc662ba-e2e8-4fac-88ff-ab1acbd4fd8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:11:14.085614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc662ba-e2e8-4fac-88ff-ab1acbd4fd8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:14.086971 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc662ba-e2e8-4fac-88ff-ab1acbd4fd8d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:14.095336 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 565/1142] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:11:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:14.101366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb436d7e-2277-4dfa-a46c-a211af88ca9d None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:14.162402 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:14.198954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb436d7e-2277-4dfa-a46c-a211af88ca9d None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:14.231205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793fccd-1a88-410d-bea7-4e7b13b48910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:14.231820 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793fccd-1a88-410d-bea7-4e7b13b48910 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:14.232096 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793fccd-1a88-410d-bea7-4e7b13b48910 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:14.232366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793fccd-1a88-410d-bea7-4e7b13b48910 None admin] PATH_INFO: `/v3/users/8ae32e67eaa04771851c7ddc9afe2e7d` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:14.245002 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f793fccd-1a88-410d-bea7-4e7b13b48910 None admin] RBAC: Authorizing `identity:get_user(user_id=8ae32e67eaa04771851c7ddc9afe2e7d)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:14.246450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f793fccd-1a88-410d-bea7-4e7b13b48910 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:14.248692 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 578/1143] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:11:14 2022] GET /identity/v3/users/8ae32e67eaa04771851c7ddc9afe2e7d => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:14.254440 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc662ba-e2e8-4fac-88ff-ab1acbd4fd8d None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:14.288853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:14.355158 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc662ba-e2e8-4fac-88ff-ab1acbd4fd8d None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:14.387613 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27241e9e-e38f-4643-b605-4e13952dd25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:14.388812 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27241e9e-e38f-4643-b605-4e13952dd25a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:14.389086 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27241e9e-e38f-4643-b605-4e13952dd25a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:14.389461 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27241e9e-e38f-4643-b605-4e13952dd25a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:14.459757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27241e9e-e38f-4643-b605-4e13952dd25a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:14.461233 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27241e9e-e38f-4643-b605-4e13952dd25a None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:14.493314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:14.556015 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 566/1144] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:11:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:11:14.562702 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793fccd-1a88-410d-bea7-4e7b13b48910 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:14.657414 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793fccd-1a88-410d-bea7-4e7b13b48910 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:14.679133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:14.691302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb2e09-821e-4bdb-9a00-7d2791f74683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:14.691945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb2e09-821e-4bdb-9a00-7d2791f74683 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:14.692223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb2e09-821e-4bdb-9a00-7d2791f74683 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:14.692490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb2e09-821e-4bdb-9a00-7d2791f74683 None admin] PATH_INFO: `/v3/projects/8a7da3381293421c87fa4ae3cde1a5cb` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:14.693601 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb2e09-821e-4bdb-9a00-7d2791f74683 None admin] RBAC: Authorizing `identity:get_project(project_id=8a7da3381293421c87fa4ae3cde1a5cb)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:14.695010 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb2e09-821e-4bdb-9a00-7d2791f74683 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:14.696596 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 579/1145] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:11:14 2022] GET /identity/v3/projects/8a7da3381293421c87fa4ae3cde1a5cb => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:14.702925 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27241e9e-e38f-4643-b605-4e13952dd25a None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:14.798634 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27241e9e-e38f-4643-b605-4e13952dd25a None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:14.831456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95073a5-ef11-4ff7-9333-451baac6c021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:14.832045 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95073a5-ef11-4ff7-9333-451baac6c021 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:14.832320 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95073a5-ef11-4ff7-9333-451baac6c021 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:14.832586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95073a5-ef11-4ff7-9333-451baac6c021 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:14.833350 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95073a5-ef11-4ff7-9333-451baac6c021 None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=8a7da3381293421c87fa4ae3cde1a5cb, user.id=8ae32e67eaa04771851c7ddc9afe2e7d) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:11:14.833656 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95073a5-ef11-4ff7-9333-451baac6c021 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:14.835065 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95073a5-ef11-4ff7-9333-451baac6c021 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:14.837760 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:14.841229 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 567/1146] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:11:14 2022] GET /identity/v3/role_assignments?user.id=8ae32e67eaa04771851c7ddc9afe2e7d&scope.project.id=8a7da3381293421c87fa4ae3cde1a5cb&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:16.932377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c0055-32d2-429e-a30e-c47aa3c587a6 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:16.932910 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c0055-32d2-429e-a30e-c47aa3c587a6 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:16.932910 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c0055-32d2-429e-a30e-c47aa3c587a6 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:16.934294 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 580/1147] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:11:16.939988 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad44e68-acba-4666-806d-b88a460938ef None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:16.940387 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad44e68-acba-4666-806d-b88a460938ef None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:16.940565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad44e68-acba-4666-806d-b88a460938ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:16.958551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-7ad44e68-acba-4666-806d-b88a460938ef None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:11:16.959718 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7ad44e68-acba-4666-806d-b88a460938ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:11:17.023400 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 568/1148] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:11:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:11:17.293651 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c0055-32d2-429e-a30e-c47aa3c587a6 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:17.392546 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c0055-32d2-429e-a30e-c47aa3c587a6 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:17.425426 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2688ef-c97a-4c79-b75e-bc262cd57ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:17.426037 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2688ef-c97a-4c79-b75e-bc262cd57ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:17.426316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2688ef-c97a-4c79-b75e-bc262cd57ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:17.426588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2688ef-c97a-4c79-b75e-bc262cd57ed6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:17.436251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2688ef-c97a-4c79-b75e-bc262cd57ed6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:11:17.440982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-ff2688ef-c97a-4c79-b75e-bc262cd57ed6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:11:17.440982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:11:17.440982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:11:17.440982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:11:17.440982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:11:17.440982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:11:17.440982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:11:17.440982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:11:17.440982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:11:17.440982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:11:17.440982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:11:17.440982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:11:17.440982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:11:17.440982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:11:17.440982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:11:17.440982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:11:17.442059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:11:17.442059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:11:17.442059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:11:17.442059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:11:17.442059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:11:17.442059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:11:17.442059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:11:17.442059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:11:17.442059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:11:17.442059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:11:17.442059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:11:17.442059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:11:17.442059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:11:17.442059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:11:17.442059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:11:17.442059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:11:17.443089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:11:17.443089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:11:17.443089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:11:17.443089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:11:17.443089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:11:17.443089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:11:17.443089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:11:17.443089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:11:17.443089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:11:17.443089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:11:17.443089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:11:17.443089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:11:17.443089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 581/1149] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:11:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:11:17.447957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad44e68-acba-4666-806d-b88a460938ef None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:17.543561 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad44e68-acba-4666-806d-b88a460938ef None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:17.571395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:17.577715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84639a35-7faf-4bdb-b5ba-0290785fb1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:17.578309 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84639a35-7faf-4bdb-b5ba-0290785fb1af None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:17.578588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84639a35-7faf-4bdb-b5ba-0290785fb1af None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:17.578858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84639a35-7faf-4bdb-b5ba-0290785fb1af None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:17.579507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84639a35-7faf-4bdb-b5ba-0290785fb1af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:11:17.579805 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84639a35-7faf-4bdb-b5ba-0290785fb1af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:17.581167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84639a35-7faf-4bdb-b5ba-0290785fb1af None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:17.588940 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 569/1150] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:11:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:17.595428 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2688ef-c97a-4c79-b75e-bc262cd57ed6 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:17.644317 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:17.691747 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2688ef-c97a-4c79-b75e-bc262cd57ed6 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:17.724429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189a0c0-7b54-464b-a032-fc65c4e5f182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:17.725074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189a0c0-7b54-464b-a032-fc65c4e5f182 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:17.725385 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189a0c0-7b54-464b-a032-fc65c4e5f182 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:17.725658 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189a0c0-7b54-464b-a032-fc65c4e5f182 None admin] PATH_INFO: `/v3/users/8ae32e67eaa04771851c7ddc9afe2e7d` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:17.726822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189a0c0-7b54-464b-a032-fc65c4e5f182 None admin] RBAC: Authorizing `identity:get_user(user_id=8ae32e67eaa04771851c7ddc9afe2e7d)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:17.728240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189a0c0-7b54-464b-a032-fc65c4e5f182 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:17.729821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 582/1151] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:11:17 2022] GET /identity/v3/users/8ae32e67eaa04771851c7ddc9afe2e7d => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:17.735899 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84639a35-7faf-4bdb-b5ba-0290785fb1af None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:17.832032 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84639a35-7faf-4bdb-b5ba-0290785fb1af None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:17.865431 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b2ab5-9b55-424a-90da-f68942be9399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:17.866058 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b2ab5-9b55-424a-90da-f68942be9399 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:17.866343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b2ab5-9b55-424a-90da-f68942be9399 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:17.866622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b2ab5-9b55-424a-90da-f68942be9399 None admin] PATH_INFO: `/v3/projects/8a7da3381293421c87fa4ae3cde1a5cb` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:17.867735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b2ab5-9b55-424a-90da-f68942be9399 None admin] RBAC: Authorizing `identity:get_project(project_id=8a7da3381293421c87fa4ae3cde1a5cb)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:17.869181 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b2ab5-9b55-424a-90da-f68942be9399 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:17.870794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 570/1152] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:11:17 2022] GET /identity/v3/projects/8a7da3381293421c87fa4ae3cde1a5cb => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:17.876595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189a0c0-7b54-464b-a032-fc65c4e5f182 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:17.972720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189a0c0-7b54-464b-a032-fc65c4e5f182 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:17.989684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:18.006800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04eb4d-a065-417e-94bf-ad84f197a7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:18.007446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04eb4d-a065-417e-94bf-ad84f197a7b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:18.007728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04eb4d-a065-417e-94bf-ad84f197a7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:18.008003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04eb4d-a065-417e-94bf-ad84f197a7b0 None admin] PATH_INFO: `/v3/projects/8a7da3381293421c87fa4ae3cde1a5cb/users/8ae32e67eaa04771851c7ddc9afe2e7d/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:18.009897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04eb4d-a065-417e-94bf-ad84f197a7b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=8a7da3381293421c87fa4ae3cde1a5cb, user_id=8ae32e67eaa04771851c7ddc9afe2e7d, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:18.011311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04eb4d-a065-417e-94bf-ad84f197a7b0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:18.024607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 583/1153] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:11:17 2022] PUT /identity/v3/projects/8a7da3381293421c87fa4ae3cde1a5cb/users/8ae32e67eaa04771851c7ddc9afe2e7d/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:11:20.132143 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c99a00-0c53-4766-bcae-8f6433f53b9e None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:20.132711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c99a00-0c53-4766-bcae-8f6433f53b9e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:20.132711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c99a00-0c53-4766-bcae-8f6433f53b9e None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:20.134082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 571/1154] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:11:20.139419 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85f711-29cb-45dc-933c-1d6aba43a4a4 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:20.139799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85f711-29cb-45dc-933c-1d6aba43a4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:20.139960 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85f711-29cb-45dc-933c-1d6aba43a4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:20.158737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-8e85f711-29cb-45dc-933c-1d6aba43a4a4 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:11:20.159906 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-8e85f711-29cb-45dc-933c-1d6aba43a4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:11:20.224105 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 584/1155] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:11:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:11:20.491273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c99a00-0c53-4766-bcae-8f6433f53b9e None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:20.551279 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:20.592776 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c99a00-0c53-4766-bcae-8f6433f53b9e None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:20.624932 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbbb8a-29dc-487c-903d-59a309390f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:20.625566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbbb8a-29dc-487c-903d-59a309390f53 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:20.625847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbbb8a-29dc-487c-903d-59a309390f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:20.626116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbbb8a-29dc-487c-903d-59a309390f53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:20.636321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dbbb8a-29dc-487c-903d-59a309390f53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:11:20.641679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-63dbbb8a-29dc-487c-903d-59a309390f53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:11:20.641679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:11:20.641679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:11:20.641679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:11:20.641679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:11:20.641679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:11:20.641679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:11:20.641679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:11:20.641679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:11:20.641679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:11:20.641679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:11:20.641679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:11:20.641679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:11:20.641679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:11:20.641679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:11:20.641679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:11:20.642761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:11:20.642761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:11:20.642761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:11:20.642761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:11:20.642761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:11:20.642761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:11:20.642761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:11:20.642761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:11:20.642761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:11:20.642761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:11:20.642761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:11:20.642761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:11:20.642761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:11:20.642761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:11:20.642761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:11:20.642761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:11:20.643943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:11:20.643943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:11:20.643943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:11:20.643943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:11:20.643943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:11:20.643943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:11:20.643943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:11:20.643943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:11:20.643943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:11:20.643943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:11:20.643943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:11:20.643943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:11:20.643943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 572/1156] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:11:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:11:20.648813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85f711-29cb-45dc-933c-1d6aba43a4a4 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:20.743210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85f711-29cb-45dc-933c-1d6aba43a4a4 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:20.752079 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:20.777317 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b066904-bd00-44fa-a16d-03a2f4fb57c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:20.777909 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b066904-bd00-44fa-a16d-03a2f4fb57c0 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:20.778183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b066904-bd00-44fa-a16d-03a2f4fb57c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:20.779085 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b066904-bd00-44fa-a16d-03a2f4fb57c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:20.779884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b066904-bd00-44fa-a16d-03a2f4fb57c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:11:20.780185 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b066904-bd00-44fa-a16d-03a2f4fb57c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:20.781562 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b066904-bd00-44fa-a16d-03a2f4fb57c0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:20.789226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 585/1157] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:11:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:20.795220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbbb8a-29dc-487c-903d-59a309390f53 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:20.892714 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbbb8a-29dc-487c-903d-59a309390f53 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:20.911360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:20.925002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d6679-0f35-41ae-9ad5-32cd5e867e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:20.925654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d6679-0f35-41ae-9ad5-32cd5e867e4f None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:20.925935 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d6679-0f35-41ae-9ad5-32cd5e867e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:20.926205 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d6679-0f35-41ae-9ad5-32cd5e867e4f None admin] PATH_INFO: `/v3/users/8ae32e67eaa04771851c7ddc9afe2e7d` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:20.927306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d6679-0f35-41ae-9ad5-32cd5e867e4f None admin] RBAC: Authorizing `identity:get_user(user_id=8ae32e67eaa04771851c7ddc9afe2e7d)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:20.928728 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d6679-0f35-41ae-9ad5-32cd5e867e4f None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:20.930310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 573/1158] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:11:20 2022] GET /identity/v3/users/8ae32e67eaa04771851c7ddc9afe2e7d => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:20.936472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b066904-bd00-44fa-a16d-03a2f4fb57c0 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:21.032777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b066904-bd00-44fa-a16d-03a2f4fb57c0 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:21.064776 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce16fd-377d-4379-8358-b89a84222ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:21.065398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce16fd-377d-4379-8358-b89a84222ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:21.065676 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce16fd-377d-4379-8358-b89a84222ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:21.065945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce16fd-377d-4379-8358-b89a84222ab4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:21.102874 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:21.130581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce16fd-377d-4379-8358-b89a84222ab4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:21.132009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ce16fd-377d-4379-8358-b89a84222ab4 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:21.225974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 586/1159] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:11:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:11:21.232106 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d6679-0f35-41ae-9ad5-32cd5e867e4f None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:21.333849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d6679-0f35-41ae-9ad5-32cd5e867e4f None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:21.366383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692a157-8183-4c26-a0b7-05f72bf463f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:21.367002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692a157-8183-4c26-a0b7-05f72bf463f6 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:21.367279 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692a157-8183-4c26-a0b7-05f72bf463f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:21.367549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692a157-8183-4c26-a0b7-05f72bf463f6 None admin] PATH_INFO: `/v3/projects/8a7da3381293421c87fa4ae3cde1a5cb` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:21.368640 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692a157-8183-4c26-a0b7-05f72bf463f6 None admin] RBAC: Authorizing `identity:get_project(project_id=8a7da3381293421c87fa4ae3cde1a5cb)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:21.370075 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692a157-8183-4c26-a0b7-05f72bf463f6 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:21.371671 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 574/1160] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:11:21 2022] GET /identity/v3/projects/8a7da3381293421c87fa4ae3cde1a5cb => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:21.378441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce16fd-377d-4379-8358-b89a84222ab4 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:21.443277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:21.472472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce16fd-377d-4379-8358-b89a84222ab4 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:21.505373 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b4560-b68f-4568-8de6-55675c0a30ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:21.505962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b4560-b68f-4568-8de6-55675c0a30ed None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:21.506237 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b4560-b68f-4568-8de6-55675c0a30ed None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:21.506505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b4560-b68f-4568-8de6-55675c0a30ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:21.507212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b4560-b68f-4568-8de6-55675c0a30ed None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=8a7da3381293421c87fa4ae3cde1a5cb, user.id=8ae32e67eaa04771851c7ddc9afe2e7d) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:11:21.507516 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b4560-b68f-4568-8de6-55675c0a30ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:21.508910 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b4560-b68f-4568-8de6-55675c0a30ed None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:21.514577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 587/1161] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:11:21 2022] GET /identity/v3/role_assignments?user.id=8ae32e67eaa04771851c7ddc9afe2e7d&scope.project.id=8a7da3381293421c87fa4ae3cde1a5cb&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:23.654438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5fbe2-9458-40da-a90c-48ed4f25a85d None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:23.655020 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5fbe2-9458-40da-a90c-48ed4f25a85d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:23.655020 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5fbe2-9458-40da-a90c-48ed4f25a85d None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:23.656312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 575/1162] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:11:23.661492 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d4ac9-3016-4373-8529-2bf92e393046 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:23.661875 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d4ac9-3016-4373-8529-2bf92e393046 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:23.662038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d4ac9-3016-4373-8529-2bf92e393046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:23.680455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-b44d4ac9-3016-4373-8529-2bf92e393046 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:11:23.681651 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b44d4ac9-3016-4373-8529-2bf92e393046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:11:23.745671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 588/1163] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:11:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:11:24.009794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5fbe2-9458-40da-a90c-48ed4f25a85d None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:24.066649 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:24.110001 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5fbe2-9458-40da-a90c-48ed4f25a85d None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:24.142985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ea512-6e54-4925-adf0-66514b18fd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:24.143577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ea512-6e54-4925-adf0-66514b18fd33 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:24.143854 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ea512-6e54-4925-adf0-66514b18fd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:24.144123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ea512-6e54-4925-adf0-66514b18fd33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:24.209643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ea512-6e54-4925-adf0-66514b18fd33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:24.211075 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ea512-6e54-4925-adf0-66514b18fd33 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:24.263726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:24.306628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 576/1164] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:11:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:11:24.313105 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d4ac9-3016-4373-8529-2bf92e393046 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:24.373337 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:24.409028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d4ac9-3016-4373-8529-2bf92e393046 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:24.442517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e912425-6252-4b0b-8836-7deda9bf5a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:24.443137 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e912425-6252-4b0b-8836-7deda9bf5a79 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:24.443413 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e912425-6252-4b0b-8836-7deda9bf5a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:24.443684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e912425-6252-4b0b-8836-7deda9bf5a79 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:24.450173 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e912425-6252-4b0b-8836-7deda9bf5a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:24.451833 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e912425-6252-4b0b-8836-7deda9bf5a79 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:24.462889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-9e912425-6252-4b0b-8836-7deda9bf5a79 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 28 12:11:24.462889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:11:24.462889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 28 12:11:24.462889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 28 12:11:24.462889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:11:24.462889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:11:24.462889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:11:24.462889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:11:24.462889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 28 12:11:24.462889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:11:24.462889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:11:24.462889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:11:24.462889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:11:24.462889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:11:24.462889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:11:24.462889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:11:24.462889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:11:24.463974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:11:24.463974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:11:24.463974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:11:24.463974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:11:24.463974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:11:24.463974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:11:24.463974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:11:24.463974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:11:24.463974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:11:24.463974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:11:24.463974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:11:24.463974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:11:24.463974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:11:24.463974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:11:24.463974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:11:24.463974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:11:24.464989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:11:24.464989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:11:24.464989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:11:24.464989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:11:24.464989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:11:24.464989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:11:24.464989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:11:24.464989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:11:24.464989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:11:24.464989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:11:24.464989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 28 12:11:24.464989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:11:24.464989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 28 12:11:24.464989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:11:24.464989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 589/1165] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 12:11:24 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:11:24.470011 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ea512-6e54-4925-adf0-66514b18fd33 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:24.567405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ea512-6e54-4925-adf0-66514b18fd33 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:24.599213 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d7da53-eed5-4f2d-89aa-590aaedc0304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:24.599802 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7da53-eed5-4f2d-89aa-590aaedc0304 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:24.600079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7da53-eed5-4f2d-89aa-590aaedc0304 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:24.600345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d7da53-eed5-4f2d-89aa-590aaedc0304 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:24.600972 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d7da53-eed5-4f2d-89aa-590aaedc0304 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:11:24.601301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d7da53-eed5-4f2d-89aa-590aaedc0304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:24.602644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d7da53-eed5-4f2d-89aa-590aaedc0304 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:24.610251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 577/1166] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Sep 28 12:11:24 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:26.704227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225ed20-1515-4f48-afb7-054482954031 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:26.704801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225ed20-1515-4f48-afb7-054482954031 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:26.704801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225ed20-1515-4f48-afb7-054482954031 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:26.706117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 590/1167] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:11:26.711524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1857b4c-1e3b-4f2f-9b08-ea09dac6c9ce None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:26.711960 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1857b4c-1e3b-4f2f-9b08-ea09dac6c9ce None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:26.712063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1857b4c-1e3b-4f2f-9b08-ea09dac6c9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:26.730269 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-d1857b4c-1e3b-4f2f-9b08-ea09dac6c9ce None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:11:26.731428 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d1857b4c-1e3b-4f2f-9b08-ea09dac6c9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:11:26.745005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:26.796104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 578/1168] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:11:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:11:27.067602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7225ed20-1515-4f48-afb7-054482954031 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:27.166360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7225ed20-1515-4f48-afb7-054482954031 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:27.189342 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:27.199908 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf003a3-14aa-4902-a9fc-75d20fc35d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:27.201309 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf003a3-14aa-4902-a9fc-75d20fc35d7c None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:27.201591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf003a3-14aa-4902-a9fc-75d20fc35d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:27.201863 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf003a3-14aa-4902-a9fc-75d20fc35d7c None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:27.202519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf003a3-14aa-4902-a9fc-75d20fc35d7c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:27.203868 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf003a3-14aa-4902-a9fc-75d20fc35d7c None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:27.218634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 591/1169] 10.222.0.60 () {60 vars in 1138 bytes} [Wed Sep 28 12:11:27 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:11:29.407355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75834f3-64a7-4ab9-805c-fd984317af99 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:29.407877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75834f3-64a7-4ab9-805c-fd984317af99 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:29.407877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75834f3-64a7-4ab9-805c-fd984317af99 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:29.409242 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 579/1170] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:11:29.414507 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483797e4-29f3-436a-8009-d5d7a2e46e7d None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:29.414944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483797e4-29f3-436a-8009-d5d7a2e46e7d None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:29.415052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483797e4-29f3-436a-8009-d5d7a2e46e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:29.433457 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-483797e4-29f3-436a-8009-d5d7a2e46e7d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:11:29.434751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-483797e4-29f3-436a-8009-d5d7a2e46e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:11:29.500145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 592/1171] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:11:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:11:29.770469 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75834f3-64a7-4ab9-805c-fd984317af99 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:29.870671 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75834f3-64a7-4ab9-805c-fd984317af99 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:29.904983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a51237c-7088-41d9-96e4-6fae2402142c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:29.905640 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51237c-7088-41d9-96e4-6fae2402142c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:29.905924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51237c-7088-41d9-96e4-6fae2402142c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:29.906198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a51237c-7088-41d9-96e4-6fae2402142c None admin] PATH_INFO: `/v3/domains/63d6cc440bc14943b7ebeef340267eab` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:29.916297 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51237c-7088-41d9-96e4-6fae2402142c None admin] RBAC: Authorizing `identity:get_domain(domain_id=63d6cc440bc14943b7ebeef340267eab)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:29.917779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a51237c-7088-41d9-96e4-6fae2402142c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:29.920271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 580/1172] 10.222.0.60 () {56 vars in 1183 bytes} [Wed Sep 28 12:11:29 2022] GET /identity/v3/domains/63d6cc440bc14943b7ebeef340267eab => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:29.926055 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483797e4-29f3-436a-8009-d5d7a2e46e7d None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:30.019660 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483797e4-29f3-436a-8009-d5d7a2e46e7d None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:30.033598 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:30.054836 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b08366-8ea5-4485-95d9-378669a8a7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:30.055433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b08366-8ea5-4485-95d9-378669a8a7bc None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:30.055710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b08366-8ea5-4485-95d9-378669a8a7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:30.055979 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b08366-8ea5-4485-95d9-378669a8a7bc None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:30.056613 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b08366-8ea5-4485-95d9-378669a8a7bc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:30.058043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b08366-8ea5-4485-95d9-378669a8a7bc None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:30.082919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 593/1173] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 12:11:29 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:11:32.243686 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54558df5-e2d6-4c52-bcea-d3169a750e3c None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:32.244194 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54558df5-e2d6-4c52-bcea-d3169a750e3c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:32.244303 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54558df5-e2d6-4c52-bcea-d3169a750e3c None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:32.245599 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 581/1174] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:11:32.251958 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baa7b6-d147-46f1-8b98-bd0d78d43e98 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:32.252385 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baa7b6-d147-46f1-8b98-bd0d78d43e98 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:32.252501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baa7b6-d147-46f1-8b98-bd0d78d43e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:32.270760 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-b4baa7b6-d147-46f1-8b98-bd0d78d43e98 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:11:32.271921 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b4baa7b6-d147-46f1-8b98-bd0d78d43e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:11:32.335095 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 594/1175] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:11:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 12:11:32.599819 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54558df5-e2d6-4c52-bcea-d3169a750e3c None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:32.613319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:32.698990 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54558df5-e2d6-4c52-bcea-d3169a750e3c None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:32.730761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87601b65-1f05-4dc0-a279-eb0976d0c3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:32.731890 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87601b65-1f05-4dc0-a279-eb0976d0c3af None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:32.732166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87601b65-1f05-4dc0-a279-eb0976d0c3af None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:32.732438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87601b65-1f05-4dc0-a279-eb0976d0c3af None admin] PATH_INFO: `/v3/domains/63d6cc440bc14943b7ebeef340267eab` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:32.733553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87601b65-1f05-4dc0-a279-eb0976d0c3af None admin] RBAC: Authorizing `identity:get_domain(domain_id=63d6cc440bc14943b7ebeef340267eab)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:32.734965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87601b65-1f05-4dc0-a279-eb0976d0c3af None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:32.736744 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 582/1176] 10.222.0.60 () {56 vars in 1183 bytes} [Wed Sep 28 12:11:32 2022] GET /identity/v3/domains/63d6cc440bc14943b7ebeef340267eab => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:32.742486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baa7b6-d147-46f1-8b98-bd0d78d43e98 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:32.832767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:32.838768 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baa7b6-d147-46f1-8b98-bd0d78d43e98 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:32.873924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b9029-fc1e-4e31-b67f-0e8666faac0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:32.874464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b9029-fc1e-4e31-b67f-0e8666faac0d None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:32.874745 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b9029-fc1e-4e31-b67f-0e8666faac0d None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:32.875019 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b9029-fc1e-4e31-b67f-0e8666faac0d None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:32.875654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b9029-fc1e-4e31-b67f-0e8666faac0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:32.877060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b9029-fc1e-4e31-b67f-0e8666faac0d None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:32.897912 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 595/1177] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:11:32 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:11:35.058302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93ecb4-04ae-4fc4-9399-b9b08708dbee None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:35.058866 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93ecb4-04ae-4fc4-9399-b9b08708dbee None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:35.058866 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93ecb4-04ae-4fc4-9399-b9b08708dbee None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:35.060206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 583/1178] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:11:35.065369 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c67b51-d584-4ca9-9d74-7481834aa40d None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:35.065803 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c67b51-d584-4ca9-9d74-7481834aa40d None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:35.065895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c67b51-d584-4ca9-9d74-7481834aa40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:35.083944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-88c67b51-d584-4ca9-9d74-7481834aa40d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:11:35.085102 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-88c67b51-d584-4ca9-9d74-7481834aa40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:11:35.148912 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 596/1179] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:11:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:11:35.417906 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93ecb4-04ae-4fc4-9399-b9b08708dbee None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:35.494356 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:35.517338 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93ecb4-04ae-4fc4-9399-b9b08708dbee None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:35.549371 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a4263-9b0b-4d3f-bef6-c29779fa4457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:35.549988 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a4263-9b0b-4d3f-bef6-c29779fa4457 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:35.550269 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a4263-9b0b-4d3f-bef6-c29779fa4457 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:35.550539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a4263-9b0b-4d3f-bef6-c29779fa4457 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:35.561217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8a4263-9b0b-4d3f-bef6-c29779fa4457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:11:35.565957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-fd8a4263-9b0b-4d3f-bef6-c29779fa4457 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:11:35.565957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:11:35.565957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:11:35.565957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:11:35.565957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:11:35.565957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:11:35.565957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:11:35.565957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:11:35.565957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:11:35.565957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:11:35.565957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:11:35.565957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:11:35.565957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:11:35.565957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:11:35.565957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:11:35.565957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:11:35.567040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:11:35.567040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:11:35.567040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:11:35.567040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:11:35.567040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:11:35.567040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:11:35.567040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:11:35.567040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:11:35.567040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:11:35.567040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:11:35.567040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:11:35.567040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:11:35.567040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:11:35.567040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:11:35.567040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:11:35.567040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:11:35.568104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:11:35.568104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:11:35.568104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:11:35.568104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:11:35.568104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:11:35.568104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:11:35.568104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:11:35.568104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:11:35.568104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:11:35.568104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:11:35.568104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:11:35.568104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:11:35.568104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 584/1180] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:11:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:11:35.573221 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c67b51-d584-4ca9-9d74-7481834aa40d None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:35.650378 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:35.667868 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c67b51-d584-4ca9-9d74-7481834aa40d None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:35.700873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae7b17-fa3c-40c3-b098-4283f8e250ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:35.701492 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae7b17-fa3c-40c3-b098-4283f8e250ce None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:35.701770 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae7b17-fa3c-40c3-b098-4283f8e250ce None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:35.702039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae7b17-fa3c-40c3-b098-4283f8e250ce None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:35.702679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae7b17-fa3c-40c3-b098-4283f8e250ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:11:35.702975 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae7b17-fa3c-40c3-b098-4283f8e250ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:35.704320 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae7b17-fa3c-40c3-b098-4283f8e250ce None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:35.711913 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 597/1181] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:11:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:35.717908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a4263-9b0b-4d3f-bef6-c29779fa4457 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:35.815715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a4263-9b0b-4d3f-bef6-c29779fa4457 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:35.847796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a1345-8fce-4e86-8a72-dc02c1499d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:35.848426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a1345-8fce-4e86-8a72-dc02c1499d9a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:35.848708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a1345-8fce-4e86-8a72-dc02c1499d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:35.849011 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a1345-8fce-4e86-8a72-dc02c1499d9a None admin] PATH_INFO: `/v3/users/ec4bc85789df41bc8717e47c278fa687` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:35.857290 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:35.864715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a1345-8fce-4e86-8a72-dc02c1499d9a None admin] RBAC: Authorizing `identity:get_user(user_id=ec4bc85789df41bc8717e47c278fa687)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:35.866190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a1345-8fce-4e86-8a72-dc02c1499d9a None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:35.867794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 585/1182] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:11:35 2022] GET /identity/v3/users/ec4bc85789df41bc8717e47c278fa687 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:35.874304 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae7b17-fa3c-40c3-b098-4283f8e250ce None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:35.969271 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae7b17-fa3c-40c3-b098-4283f8e250ce None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:36.001867 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52bc0f-6836-4c85-833c-0ef7e380de77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:36.002463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52bc0f-6836-4c85-833c-0ef7e380de77 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:36.002743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52bc0f-6836-4c85-833c-0ef7e380de77 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:36.003028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52bc0f-6836-4c85-833c-0ef7e380de77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:36.016668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:36.067911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52bc0f-6836-4c85-833c-0ef7e380de77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:36.069370 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52bc0f-6836-4c85-833c-0ef7e380de77 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:36.162388 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 598/1183] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:11:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:11:36.168259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a1345-8fce-4e86-8a72-dc02c1499d9a None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:36.264767 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a1345-8fce-4e86-8a72-dc02c1499d9a None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:36.297610 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4ba0f-c5b6-4bbc-b4f9-16249ef26bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:36.298227 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4ba0f-c5b6-4bbc-b4f9-16249ef26bee None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:36.298506 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4ba0f-c5b6-4bbc-b4f9-16249ef26bee None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:36.298779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4ba0f-c5b6-4bbc-b4f9-16249ef26bee None admin] PATH_INFO: `/v3/projects/703eebb069684255a2fc662f948c9468` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:36.299869 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4ba0f-c5b6-4bbc-b4f9-16249ef26bee None admin] RBAC: Authorizing `identity:get_project(project_id=703eebb069684255a2fc662f948c9468)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:36.301317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4ba0f-c5b6-4bbc-b4f9-16249ef26bee None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:36.302914 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 586/1184] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:11:36 2022] GET /identity/v3/projects/703eebb069684255a2fc662f948c9468 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:36.309247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52bc0f-6836-4c85-833c-0ef7e380de77 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:36.353223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:36.405418 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52bc0f-6836-4c85-833c-0ef7e380de77 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:36.437961 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f3c4e-5b60-4994-b2b1-9ea67a7f6b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:36.438551 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f3c4e-5b60-4994-b2b1-9ea67a7f6b09 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:36.438829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f3c4e-5b60-4994-b2b1-9ea67a7f6b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:36.439099 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f3c4e-5b60-4994-b2b1-9ea67a7f6b09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:36.439808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f3c4e-5b60-4994-b2b1-9ea67a7f6b09 None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=703eebb069684255a2fc662f948c9468, user.id=ec4bc85789df41bc8717e47c278fa687) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:11:36.440113 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f3c4e-5b60-4994-b2b1-9ea67a7f6b09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:36.441544 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f3c4e-5b60-4994-b2b1-9ea67a7f6b09 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:36.446658 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 599/1185] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:11:36 2022] GET /identity/v3/role_assignments?user.id=ec4bc85789df41bc8717e47c278fa687&scope.project.id=703eebb069684255a2fc662f948c9468&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:38.540572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0486b-d115-49c1-9e43-b9f32558789d None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:38.541115 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0486b-d115-49c1-9e43-b9f32558789d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:38.541115 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0486b-d115-49c1-9e43-b9f32558789d None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:38.542588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 587/1186] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:11:38.548360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fa80a-2f75-4d34-a2e6-c8af4622d55c None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:38.548808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fa80a-2f75-4d34-a2e6-c8af4622d55c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:38.548911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fa80a-2f75-4d34-a2e6-c8af4622d55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:38.566574 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-979fa80a-2f75-4d34-a2e6-c8af4622d55c None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:11:38.567733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-979fa80a-2f75-4d34-a2e6-c8af4622d55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:11:38.631273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 600/1187] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:11:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:11:38.899509 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0486b-d115-49c1-9e43-b9f32558789d None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:38.962556 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:39.001275 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0486b-d115-49c1-9e43-b9f32558789d None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:39.034534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd66f58-db0f-4c21-bc3e-8e1bec16de13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:39.035154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd66f58-db0f-4c21-bc3e-8e1bec16de13 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:39.035441 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd66f58-db0f-4c21-bc3e-8e1bec16de13 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:39.035715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd66f58-db0f-4c21-bc3e-8e1bec16de13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:39.045736 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd66f58-db0f-4c21-bc3e-8e1bec16de13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:11:39.050558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-3dd66f58-db0f-4c21-bc3e-8e1bec16de13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:11:39.050558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:11:39.050558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:11:39.050558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:11:39.050558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:11:39.050558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:11:39.050558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:11:39.050558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:11:39.050558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:11:39.050558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:11:39.050558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:11:39.050558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:11:39.050558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:11:39.050558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:11:39.050558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:11:39.050558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:11:39.051644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:11:39.051644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:11:39.051644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:11:39.051644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:11:39.051644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:11:39.051644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:11:39.051644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:11:39.051644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:11:39.051644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:11:39.051644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:11:39.051644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:11:39.051644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:11:39.051644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:11:39.051644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:11:39.051644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:11:39.051644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:11:39.052598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:11:39.052598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:11:39.052598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:11:39.052598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:11:39.052598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:11:39.052598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:11:39.052598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:11:39.052598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:11:39.052598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:11:39.052598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:11:39.052598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:11:39.052598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:11:39.052598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 588/1188] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:11:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:11:39.057668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fa80a-2f75-4d34-a2e6-c8af4622d55c None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:39.096049 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:39.154536 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fa80a-2f75-4d34-a2e6-c8af4622d55c None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:39.187844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34586701-8cb5-4db1-b47b-f8ab9fe898cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:39.188453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34586701-8cb5-4db1-b47b-f8ab9fe898cf None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:39.188734 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34586701-8cb5-4db1-b47b-f8ab9fe898cf None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:39.189005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34586701-8cb5-4db1-b47b-f8ab9fe898cf None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:39.189686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34586701-8cb5-4db1-b47b-f8ab9fe898cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:11:39.189989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34586701-8cb5-4db1-b47b-f8ab9fe898cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:39.191339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34586701-8cb5-4db1-b47b-f8ab9fe898cf None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:39.199258 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 601/1189] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:11:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:39.205617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd66f58-db0f-4c21-bc3e-8e1bec16de13 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:39.301717 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd66f58-db0f-4c21-bc3e-8e1bec16de13 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:39.324625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:39.335681 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d1533-9345-4d39-ae2c-e9567452602d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:39.336294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d1533-9345-4d39-ae2c-e9567452602d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:39.336573 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d1533-9345-4d39-ae2c-e9567452602d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:39.336845 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d1533-9345-4d39-ae2c-e9567452602d None admin] PATH_INFO: `/v3/users/ec4bc85789df41bc8717e47c278fa687` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:39.337986 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d1533-9345-4d39-ae2c-e9567452602d None admin] RBAC: Authorizing `identity:get_user(user_id=ec4bc85789df41bc8717e47c278fa687)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:39.339397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d1533-9345-4d39-ae2c-e9567452602d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:39.341084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 589/1190] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:11:39 2022] GET /identity/v3/users/ec4bc85789df41bc8717e47c278fa687 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:39.346720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34586701-8cb5-4db1-b47b-f8ab9fe898cf None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:39.439361 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:39.442158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34586701-8cb5-4db1-b47b-f8ab9fe898cf None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:39.474064 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c809bad-689a-49e7-830f-2ccd0829ca49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:39.474708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c809bad-689a-49e7-830f-2ccd0829ca49 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:39.474984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c809bad-689a-49e7-830f-2ccd0829ca49 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:39.475255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c809bad-689a-49e7-830f-2ccd0829ca49 None admin] PATH_INFO: `/v3/projects/703eebb069684255a2fc662f948c9468` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:39.476346 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c809bad-689a-49e7-830f-2ccd0829ca49 None admin] RBAC: Authorizing `identity:get_project(project_id=703eebb069684255a2fc662f948c9468)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:39.477807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c809bad-689a-49e7-830f-2ccd0829ca49 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:39.479391 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 602/1191] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:11:39 2022] GET /identity/v3/projects/703eebb069684255a2fc662f948c9468 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:39.485918 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d1533-9345-4d39-ae2c-e9567452602d None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:39.582221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d1533-9345-4d39-ae2c-e9567452602d None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:39.615319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2369b-b20d-4463-a87c-5560d8163c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:39.615961 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2369b-b20d-4463-a87c-5560d8163c0d None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:39.616243 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2369b-b20d-4463-a87c-5560d8163c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:39.616514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2369b-b20d-4463-a87c-5560d8163c0d None admin] PATH_INFO: `/v3/projects/703eebb069684255a2fc662f948c9468/users/ec4bc85789df41bc8717e47c278fa687/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:39.618412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2369b-b20d-4463-a87c-5560d8163c0d None admin] RBAC: Authorizing `identity:create_grant(project_id=703eebb069684255a2fc662f948c9468, user_id=ec4bc85789df41bc8717e47c278fa687, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:39.619829 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2369b-b20d-4463-a87c-5560d8163c0d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:39.632885 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 590/1192] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:11:39 2022] PUT /identity/v3/projects/703eebb069684255a2fc662f948c9468/users/ec4bc85789df41bc8717e47c278fa687/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:11:41.752064 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2240d-0615-450a-b2be-73fc348c5e53 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:41.752625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2240d-0615-450a-b2be-73fc348c5e53 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:41.752625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2240d-0615-450a-b2be-73fc348c5e53 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:41.753964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 603/1193] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:11:41.759088 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90425ddc-2c3c-4961-96f2-acb56f4fdc1c None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:41.759516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90425ddc-2c3c-4961-96f2-acb56f4fdc1c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:41.759642 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90425ddc-2c3c-4961-96f2-acb56f4fdc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:41.777559 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-90425ddc-2c3c-4961-96f2-acb56f4fdc1c None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:11:41.778753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-90425ddc-2c3c-4961-96f2-acb56f4fdc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:11:41.825691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:41.845599 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 591/1194] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:11:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:11:42.121665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2240d-0615-450a-b2be-73fc348c5e53 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:42.221660 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2240d-0615-450a-b2be-73fc348c5e53 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:42.254250 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2cf01-15bb-48be-9a67-3bb86db95185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:42.254864 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2cf01-15bb-48be-9a67-3bb86db95185 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:42.255143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2cf01-15bb-48be-9a67-3bb86db95185 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:42.255412 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2cf01-15bb-48be-9a67-3bb86db95185 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:42.265694 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2cf01-15bb-48be-9a67-3bb86db95185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:11:42.270409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-d7a2cf01-15bb-48be-9a67-3bb86db95185 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:11:42.270409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:11:42.270409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:11:42.270409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:11:42.270409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:11:42.270409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:11:42.270409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:11:42.270409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:11:42.270409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:11:42.270409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:11:42.270409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:11:42.270409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:11:42.270409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:11:42.270409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:11:42.270409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:11:42.270409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:11:42.271489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:11:42.271489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:11:42.271489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:11:42.271489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:11:42.271489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:11:42.271489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:11:42.271489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:11:42.271489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:11:42.271489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:11:42.271489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:11:42.271489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:11:42.271489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:11:42.271489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:11:42.271489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:11:42.271489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:11:42.271489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:11:42.272334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:11:42.272334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:11:42.272334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:11:42.272334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:11:42.272334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:11:42.272334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:11:42.272334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:11:42.272334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:11:42.272334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:11:42.272334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:11:42.272334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:11:42.272334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:11:42.272334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 604/1195] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:11:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:11:42.278212 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90425ddc-2c3c-4961-96f2-acb56f4fdc1c None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:42.373514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90425ddc-2c3c-4961-96f2-acb56f4fdc1c None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:42.406834 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c52ca4-73df-46eb-ad5e-d42b7a1198da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:42.407429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c52ca4-73df-46eb-ad5e-d42b7a1198da None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:42.407709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c52ca4-73df-46eb-ad5e-d42b7a1198da None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:42.407980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c52ca4-73df-46eb-ad5e-d42b7a1198da None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:42.408635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c52ca4-73df-46eb-ad5e-d42b7a1198da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:11:42.408935 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c52ca4-73df-46eb-ad5e-d42b7a1198da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:42.410331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c52ca4-73df-46eb-ad5e-d42b7a1198da None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:42.418030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 592/1196] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:11:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:42.423904 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2cf01-15bb-48be-9a67-3bb86db95185 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:42.451923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:42.520651 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2cf01-15bb-48be-9a67-3bb86db95185 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:42.552868 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cad63b-0257-4b66-97bf-e0f8b673823a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:42.553514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cad63b-0257-4b66-97bf-e0f8b673823a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:42.553793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cad63b-0257-4b66-97bf-e0f8b673823a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:42.554065 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cad63b-0257-4b66-97bf-e0f8b673823a None admin] PATH_INFO: `/v3/users/ec4bc85789df41bc8717e47c278fa687` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:42.555279 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cad63b-0257-4b66-97bf-e0f8b673823a None admin] RBAC: Authorizing `identity:get_user(user_id=ec4bc85789df41bc8717e47c278fa687)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:42.556908 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cad63b-0257-4b66-97bf-e0f8b673823a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:42.558495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 605/1197] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Sep 28 12:11:42 2022] GET /identity/v3/users/ec4bc85789df41bc8717e47c278fa687 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:42.564333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c52ca4-73df-46eb-ad5e-d42b7a1198da None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:42.613846 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:42.667063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c52ca4-73df-46eb-ad5e-d42b7a1198da None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:42.700158 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab54955-b2ef-47c2-9cbb-459c8aaf3275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:42.700757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab54955-b2ef-47c2-9cbb-459c8aaf3275 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:42.701035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab54955-b2ef-47c2-9cbb-459c8aaf3275 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:42.701337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab54955-b2ef-47c2-9cbb-459c8aaf3275 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:42.768534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab54955-b2ef-47c2-9cbb-459c8aaf3275 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:42.770011 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab54955-b2ef-47c2-9cbb-459c8aaf3275 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:42.812047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:42.865051 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 593/1198] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:11:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:11:42.870932 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cad63b-0257-4b66-97bf-e0f8b673823a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:42.962641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:42.968695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cad63b-0257-4b66-97bf-e0f8b673823a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:43.000801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6bc1b-9dee-4925-93dd-b57b42ae0900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:43.001427 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6bc1b-9dee-4925-93dd-b57b42ae0900 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:43.001701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6bc1b-9dee-4925-93dd-b57b42ae0900 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:43.001969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6bc1b-9dee-4925-93dd-b57b42ae0900 None admin] PATH_INFO: `/v3/projects/703eebb069684255a2fc662f948c9468` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:43.003045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6bc1b-9dee-4925-93dd-b57b42ae0900 None admin] RBAC: Authorizing `identity:get_project(project_id=703eebb069684255a2fc662f948c9468)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:43.004455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6bc1b-9dee-4925-93dd-b57b42ae0900 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:43.006093 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 606/1199] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Sep 28 12:11:42 2022] GET /identity/v3/projects/703eebb069684255a2fc662f948c9468 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:43.012889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab54955-b2ef-47c2-9cbb-459c8aaf3275 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:11:43.111344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab54955-b2ef-47c2-9cbb-459c8aaf3275 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:11:43.144313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedd282-220c-4e65-9184-3265304fa43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:11:43.144909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedd282-220c-4e65-9184-3265304fa43f None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:11:43.145221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedd282-220c-4e65-9184-3265304fa43f None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:11:43.145496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedd282-220c-4e65-9184-3265304fa43f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:11:43.146213 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedd282-220c-4e65-9184-3265304fa43f None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=703eebb069684255a2fc662f948c9468, user.id=ec4bc85789df41bc8717e47c278fa687) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:11:43.146520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedd282-220c-4e65-9184-3265304fa43f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:11:43.147938 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedd282-220c-4e65-9184-3265304fa43f None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:11:43.153784 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 594/1200] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:11:43 2022] GET /identity/v3/role_assignments?user.id=ec4bc85789df41bc8717e47c278fa687&scope.project.id=703eebb069684255a2fc662f948c9468&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:11:52.996436 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=18 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:11:53.150964 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=96 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:39.166893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efabad-177e-45a5-becd-af30608473da None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:39.167508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efabad-177e-45a5-becd-af30608473da None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:39.167598 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efabad-177e-45a5-becd-af30608473da None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:39.169230 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 607/1201] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:12:39.175508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f4bb6-48e4-4bc4-8234-294a82220990 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:39.176076 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f4bb6-48e4-4bc4-8234-294a82220990 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:39.176076 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f4bb6-48e4-4bc4-8234-294a82220990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:39.202836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-ca8f4bb6-48e4-4bc4-8234-294a82220990 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:12:39.204053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-ca8f4bb6-48e4-4bc4-8234-294a82220990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:12:39.314485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 595/1202] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:12:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:12:39.659843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efabad-177e-45a5-becd-af30608473da None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:39.791569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efabad-177e-45a5-becd-af30608473da None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:39.837731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0616231-35b1-4a30-8d68-6d8fa8ea253f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:39.838479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0616231-35b1-4a30-8d68-6d8fa8ea253f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:39.838781 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0616231-35b1-4a30-8d68-6d8fa8ea253f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:39.839075 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0616231-35b1-4a30-8d68-6d8fa8ea253f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:39.850985 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0616231-35b1-4a30-8d68-6d8fa8ea253f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:39.852804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0616231-35b1-4a30-8d68-6d8fa8ea253f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:39.875505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-d0616231-35b1-4a30-8d68-6d8fa8ea253f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:39.875505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:39.875505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:39.875505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:39.875505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:39.875505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:39.875505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:39.875505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:39.875505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:39.875505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:39.875505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:39.875505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:39.875505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:12:39.875505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:12:39.875505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:12:39.875505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:12:39.887101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:39.887101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:39.887101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:39.887101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:39.887101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:39.887101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:39.887101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:39.887101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:39.887101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:39.887101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:39.887101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:39.887101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:39.887101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:39.887101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:39.887101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:39.887101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:39.887101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:12:39.891670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:12:39.891670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:39.891670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:12:39.891670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 608/1203] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:39.892144 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f4bb6-48e4-4bc4-8234-294a82220990 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:40.063097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f4bb6-48e4-4bc4-8234-294a82220990 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:40.096460 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:40.102796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9c474-c95c-41ed-9921-0f5fbd0c257b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:40.103413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9c474-c95c-41ed-9921-0f5fbd0c257b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:40.103693 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9c474-c95c-41ed-9921-0f5fbd0c257b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:40.103962 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9c474-c95c-41ed-9921-0f5fbd0c257b None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:40.105258 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9c474-c95c-41ed-9921-0f5fbd0c257b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:40.105842 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9c474-c95c-41ed-9921-0f5fbd0c257b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:40.108350 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9c474-c95c-41ed-9921-0f5fbd0c257b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:40.125170 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 596/1204] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:12:40.127718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0616231-35b1-4a30-8d68-6d8fa8ea253f None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:40.204870 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:40.297474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0616231-35b1-4a30-8d68-6d8fa8ea253f None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:40.341793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e793575-40b1-4218-bf6a-423a416ca53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:40.342751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e793575-40b1-4218-bf6a-423a416ca53a None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:40.343215 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e793575-40b1-4218-bf6a-423a416ca53a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:40.353179 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e793575-40b1-4218-bf6a-423a416ca53a None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:40.353179 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e793575-40b1-4218-bf6a-423a416ca53a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:40.353179 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e793575-40b1-4218-bf6a-423a416ca53a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:40.382556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 609/1205] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Sep 28 12:12:40 2022] POST /identity/v3/users => generated 251 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:12:42.700065 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf15407-86a2-4b16-a15b-1da25d164110 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:42.700634 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf15407-86a2-4b16-a15b-1da25d164110 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:42.700634 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf15407-86a2-4b16-a15b-1da25d164110 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:42.702134 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 597/1206] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:12:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:12:42.707590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda320b-6229-42f8-b848-89b1c659bdaa None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:42.711517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda320b-6229-42f8-b848-89b1c659bdaa None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:42.711517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda320b-6229-42f8-b848-89b1c659bdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:42.728008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-4bda320b-6229-42f8-b848-89b1c659bdaa None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:12:42.729359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-4bda320b-6229-42f8-b848-89b1c659bdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:12:42.805346 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 610/1207] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:12:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:12:43.096847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf15407-86a2-4b16-a15b-1da25d164110 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:43.218593 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf15407-86a2-4b16-a15b-1da25d164110 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:43.259720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45572da-c9f4-4558-b070-b5d2dc7ea080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:43.260382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45572da-c9f4-4558-b070-b5d2dc7ea080 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:43.260679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45572da-c9f4-4558-b070-b5d2dc7ea080 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:43.261758 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45572da-c9f4-4558-b070-b5d2dc7ea080 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:43.273354 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45572da-c9f4-4558-b070-b5d2dc7ea080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:12:43.278539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-e45572da-c9f4-4558-b070-b5d2dc7ea080 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:12:43.278539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:43.278539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:43.278539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:43.278539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:43.278539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:43.278539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:43.278539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:43.278539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:43.278539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:43.278539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:43.278539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:43.278539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:12:43.278539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:12:43.278539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:12:43.278539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:12:43.279518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:12:43.279518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:12:43.279518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:12:43.279518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:12:43.279518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:43.279518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:43.279518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:43.279518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:43.279518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:43.279518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:43.279518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:43.279518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:43.279518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:43.279518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:43.279518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:43.279518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:43.281929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:43.281929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:43.281929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:43.281929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:43.281929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:12:43.281929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:12:43.281929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:12:43.281929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:12:43.281929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:12:43.281929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:12:43.281929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:12:43.281929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:12:43.281929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 598/1208] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:12:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:43.286227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda320b-6229-42f8-b848-89b1c659bdaa None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:43.330229 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:43.409367 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda320b-6229-42f8-b848-89b1c659bdaa None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:43.447247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27463e53-bf77-4b67-b84f-8bfc4357a2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:43.447851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27463e53-bf77-4b67-b84f-8bfc4357a2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:43.448141 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27463e53-bf77-4b67-b84f-8bfc4357a2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:43.448442 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27463e53-bf77-4b67-b84f-8bfc4357a2a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:43.449316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27463e53-bf77-4b67-b84f-8bfc4357a2a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:43.449623 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27463e53-bf77-4b67-b84f-8bfc4357a2a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:43.451118 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27463e53-bf77-4b67-b84f-8bfc4357a2a1 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:43.459817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 611/1209] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:12:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:12:43.466120 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45572da-c9f4-4558-b070-b5d2dc7ea080 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:43.522977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:43.586912 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45572da-c9f4-4558-b070-b5d2dc7ea080 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:43.626989 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869e179-a12e-4ab8-84b0-c6315c0e15ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:43.627656 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869e179-a12e-4ab8-84b0-c6315c0e15ca None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:43.628017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869e179-a12e-4ab8-84b0-c6315c0e15ca None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:43.628234 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869e179-a12e-4ab8-84b0-c6315c0e15ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:43.642732 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869e179-a12e-4ab8-84b0-c6315c0e15ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:43.644187 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869e179-a12e-4ab8-84b0-c6315c0e15ca None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:43.661030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-a869e179-a12e-4ab8-84b0-c6315c0e15ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:43.661030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:43.661030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:43.661030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:43.661030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:43.661030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:43.661030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:43.661030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:43.661030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:43.661030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:43.661030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:43.661030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:43.661030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:12:43.661030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:12:43.661030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:12:43.661030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:12:43.662235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:43.662235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:43.662235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:43.662235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:43.662235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:43.662235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:43.662235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:43.662235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:43.662235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:43.662235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:43.662235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:43.662235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:43.662235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:43.662235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:43.662235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:43.662235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:43.662235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:12:43.663147 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:12:43.663147 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:43.663147 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:12:43.663147 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 599/1210] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:43.668506 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27463e53-bf77-4b67-b84f-8bfc4357a2a1 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:43.774895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:43.778338 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27463e53-bf77-4b67-b84f-8bfc4357a2a1 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:43.819356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8e869-41de-42fe-a57a-ae63346be67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:43.821395 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8e869-41de-42fe-a57a-ae63346be67b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:43.821395 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8e869-41de-42fe-a57a-ae63346be67b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:43.821395 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8e869-41de-42fe-a57a-ae63346be67b None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:43.821395 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8e869-41de-42fe-a57a-ae63346be67b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:43.821909 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8e869-41de-42fe-a57a-ae63346be67b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:43.822880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8e869-41de-42fe-a57a-ae63346be67b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:43.837584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 612/1211] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:12:43.844358 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869e179-a12e-4ab8-84b0-c6315c0e15ca None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:43.981781 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869e179-a12e-4ab8-84b0-c6315c0e15ca None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:43.983775 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:44.027402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e95dc0-5d37-41e3-bd80-268af6f883cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:44.028226 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e95dc0-5d37-41e3-bd80-268af6f883cb None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:44.028618 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e95dc0-5d37-41e3-bd80-268af6f883cb None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:44.028994 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e95dc0-5d37-41e3-bd80-268af6f883cb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:44.038381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e95dc0-5d37-41e3-bd80-268af6f883cb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:44.040166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e95dc0-5d37-41e3-bd80-268af6f883cb None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:44.053710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-e8e95dc0-5d37-41e3-bd80-268af6f883cb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 12:12:44.053710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:44.053710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:44.053710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:44.053710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:44.053710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:44.053710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:44.053710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:44.053710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:44.053710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:44.053710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:44.053710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:44.053710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:12:44.053710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:12:44.053710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:12:44.053710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:44.054965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:44.056139 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:44.056139 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:44.056139 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:12:44.056139 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:12:44.056139 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:12:44.056139 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:12:44.056139 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:12:44.056139 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:12:44.056139 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 12:12:44.056139 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:12:44.056751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 600/1212] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:12:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:12:44.062339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8e869-41de-42fe-a57a-ae63346be67b None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:44.183430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8e869-41de-42fe-a57a-ae63346be67b None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:44.221656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649186bd-bcb3-4c97-8447-4b72035cf59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:44.222319 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649186bd-bcb3-4c97-8447-4b72035cf59f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:44.222616 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649186bd-bcb3-4c97-8447-4b72035cf59f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:44.222933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649186bd-bcb3-4c97-8447-4b72035cf59f None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:44.223923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649186bd-bcb3-4c97-8447-4b72035cf59f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:44.224393 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649186bd-bcb3-4c97-8447-4b72035cf59f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:44.226477 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649186bd-bcb3-4c97-8447-4b72035cf59f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:44.247383 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 613/1213] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Sep 28 12:12:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:12:44.252920 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e95dc0-5d37-41e3-bd80-268af6f883cb None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:44.381614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e95dc0-5d37-41e3-bd80-268af6f883cb None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:44.427852 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65caa9-0a5a-4ceb-8294-81c9e98fb8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:44.428907 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65caa9-0a5a-4ceb-8294-81c9e98fb8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:44.429350 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65caa9-0a5a-4ceb-8294-81c9e98fb8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:44.429682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65caa9-0a5a-4ceb-8294-81c9e98fb8d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:44.472637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:44.515539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da65caa9-0a5a-4ceb-8294-81c9e98fb8d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:44.516975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da65caa9-0a5a-4ceb-8294-81c9e98fb8d5 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:44.631889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 601/1214] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:12:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:12:44.638978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649186bd-bcb3-4c97-8447-4b72035cf59f None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:44.658579 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:44.757925 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649186bd-bcb3-4c97-8447-4b72035cf59f None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:44.799466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d078d4-63fb-46f4-972c-13db997942c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:44.800109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d078d4-63fb-46f4-972c-13db997942c3 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:44.800429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d078d4-63fb-46f4-972c-13db997942c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:44.800711 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d078d4-63fb-46f4-972c-13db997942c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:44.813655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d078d4-63fb-46f4-972c-13db997942c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:44.815340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d078d4-63fb-46f4-972c-13db997942c3 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:44.836006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-a9d078d4-63fb-46f4-972c-13db997942c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:44.836006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:44.836006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:44.836006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:44.836006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:44.836006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:44.836006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:44.836006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:44.836006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:44.836006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:44.836006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:44.836006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:44.836006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:12:44.836006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:12:44.836006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:12:44.836006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:12:44.837068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:44.837068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:44.837068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:44.837068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:44.837068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:44.837068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:44.837068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:44.837068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:44.837068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:44.837068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:44.837068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:44.837068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:44.837068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:44.837068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:44.837068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:44.837068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:44.837068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:12:44.838126 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:12:44.838126 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:44.838126 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:12:44.838126 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 614/1215] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:12:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:44.844548 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65caa9-0a5a-4ceb-8294-81c9e98fb8d5 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:44.923482 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:44.969867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65caa9-0a5a-4ceb-8294-81c9e98fb8d5 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:45.008875 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3aa7f0-affe-4bb5-aaf5-d11598b73f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:45.009487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3aa7f0-affe-4bb5-aaf5-d11598b73f0d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:45.009766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3aa7f0-affe-4bb5-aaf5-d11598b73f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:45.010064 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3aa7f0-affe-4bb5-aaf5-d11598b73f0d None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:45.010804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3aa7f0-affe-4bb5-aaf5-d11598b73f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:45.011158 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3aa7f0-affe-4bb5-aaf5-d11598b73f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:45.012738 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3aa7f0-affe-4bb5-aaf5-d11598b73f0d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:45.026874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 602/1216] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:12:45.032813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d078d4-63fb-46f4-972c-13db997942c3 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:45.124019 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:45.157295 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d078d4-63fb-46f4-972c-13db997942c3 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:45.195764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac7c16-c16c-4e2f-8f64-fdc978d9380b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:45.196340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac7c16-c16c-4e2f-8f64-fdc978d9380b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:45.196621 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac7c16-c16c-4e2f-8f64-fdc978d9380b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:45.196935 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac7c16-c16c-4e2f-8f64-fdc978d9380b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:45.206937 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac7c16-c16c-4e2f-8f64-fdc978d9380b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:45.208375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac7c16-c16c-4e2f-8f64-fdc978d9380b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:45.220630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-b6ac7c16-c16c-4e2f-8f64-fdc978d9380b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:12:45.220630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:45.220630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:45.220630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:45.220630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:45.220630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:45.220630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:45.220630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:45.220630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:45.220630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:45.220630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:45.220630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:45.220630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:12:45.220630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:12:45.220630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:12:45.221765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:12:45.221765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:45.221765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:45.221765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:45.221765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:45.221765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:45.221765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:45.221765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:45.221765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:45.221765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:45.221765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:45.221765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:45.221765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:45.221765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:45.221765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:45.221765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:45.221765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:45.222860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:12:45.222860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:12:45.222860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:12:45.222860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:12:45.222860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:12:45.222860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:12:45.222860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:12:45.222860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:12:45.225293 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 615/1217] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:12:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:45.228051 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3aa7f0-affe-4bb5-aaf5-d11598b73f0d None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:45.342796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3aa7f0-affe-4bb5-aaf5-d11598b73f0d None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:45.367833 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:45.381577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32adc16c-847f-4e93-915b-79f0da7c72b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:45.382196 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adc16c-847f-4e93-915b-79f0da7c72b2 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:45.382497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adc16c-847f-4e93-915b-79f0da7c72b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:45.382797 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adc16c-847f-4e93-915b-79f0da7c72b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:45.383590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32adc16c-847f-4e93-915b-79f0da7c72b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:45.383897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32adc16c-847f-4e93-915b-79f0da7c72b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:45.385351 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32adc16c-847f-4e93-915b-79f0da7c72b2 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:45.398387 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 603/1218] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:12:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:12:45.404564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac7c16-c16c-4e2f-8f64-fdc978d9380b None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:45.523072 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac7c16-c16c-4e2f-8f64-fdc978d9380b None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:45.554717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:45.561949 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c5db6-31e0-4cd2-ac53-9c50f4862de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:45.562552 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c5db6-31e0-4cd2-ac53-9c50f4862de1 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:45.562833 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c5db6-31e0-4cd2-ac53-9c50f4862de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:45.563099 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c5db6-31e0-4cd2-ac53-9c50f4862de1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:45.563812 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c5db6-31e0-4cd2-ac53-9c50f4862de1 None admin] RBAC: Adding query filter params (role.id=60963384f3e54d9cbfbc8d12023a6cd9, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=9a2322b1a76d40e49545f5677e247489) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:45.564113 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c5db6-31e0-4cd2-ac53-9c50f4862de1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:45.565561 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c5db6-31e0-4cd2-ac53-9c50f4862de1 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:45.571178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 616/1219] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:12:45 2022] GET /identity/v3/role_assignments?user.id=9a2322b1a76d40e49545f5677e247489&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=60963384f3e54d9cbfbc8d12023a6cd9 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:12:47.752737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea455a8-b6db-45e5-a537-4a16d418d25e None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:47.753314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea455a8-b6db-45e5-a537-4a16d418d25e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:47.753434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea455a8-b6db-45e5-a537-4a16d418d25e None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:47.755142 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 604/1220] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:12:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:12:47.761682 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a7220-8e01-4ef6-936f-98b4bbebe5fd None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:47.762124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a7220-8e01-4ef6-936f-98b4bbebe5fd None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:47.762371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a7220-8e01-4ef6-936f-98b4bbebe5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:47.780972 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-e85a7220-8e01-4ef6-936f-98b4bbebe5fd None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:12:47.782171 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e85a7220-8e01-4ef6-936f-98b4bbebe5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:12:47.855652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 617/1221] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:12:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:12:48.177002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea455a8-b6db-45e5-a537-4a16d418d25e None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:48.294216 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea455a8-b6db-45e5-a537-4a16d418d25e None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:48.335210 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6ac88-da2c-43ee-9266-b0921c25dfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:48.335945 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6ac88-da2c-43ee-9266-b0921c25dfe4 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:48.336287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6ac88-da2c-43ee-9266-b0921c25dfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:48.336565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6ac88-da2c-43ee-9266-b0921c25dfe4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:48.348598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd6ac88-da2c-43ee-9266-b0921c25dfe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:12:48.353470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-0cd6ac88-da2c-43ee-9266-b0921c25dfe4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:12:48.353470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:48.353470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:48.353470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:48.353470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:48.353470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:48.353470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:48.353470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:48.353470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:48.353470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:48.353470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:48.353470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:48.353470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:12:48.353470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:12:48.353470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:12:48.353470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:12:48.354617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:12:48.354617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:12:48.354617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:12:48.354617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:12:48.354617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:48.354617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:48.354617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:48.354617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:48.354617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:48.354617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:48.354617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:48.354617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:48.354617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:48.354617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:48.354617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:48.354617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:48.355838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:48.355838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:48.355838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:48.355838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:48.355838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:12:48.355838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:12:48.355838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:12:48.355838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:12:48.355838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:12:48.355838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:12:48.355838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:12:48.355838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:12:48.355838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 605/1222] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:12:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:48.361173 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a7220-8e01-4ef6-936f-98b4bbebe5fd None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:48.466574 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a7220-8e01-4ef6-936f-98b4bbebe5fd None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:48.494310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:48.506362 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101859e9-c74a-41e7-9757-bdb3c9e18f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:48.506977 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101859e9-c74a-41e7-9757-bdb3c9e18f44 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:48.507255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101859e9-c74a-41e7-9757-bdb3c9e18f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:48.507525 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101859e9-c74a-41e7-9757-bdb3c9e18f44 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:48.508184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101859e9-c74a-41e7-9757-bdb3c9e18f44 None admin] RBAC: Adding query filter params (name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:48.508487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101859e9-c74a-41e7-9757-bdb3c9e18f44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:48.509951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101859e9-c74a-41e7-9757-bdb3c9e18f44 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:48.521514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 618/1223] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:12:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:12:48.527439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6ac88-da2c-43ee-9266-b0921c25dfe4 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:48.577116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:48.646942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6ac88-da2c-43ee-9266-b0921c25dfe4 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:48.688394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a3820-2819-49ca-8e07-bab7e72e3c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:48.689232 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a3820-2819-49ca-8e07-bab7e72e3c3b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:48.689603 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a3820-2819-49ca-8e07-bab7e72e3c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:48.689886 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a3820-2819-49ca-8e07-bab7e72e3c3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:48.702756 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a3820-2819-49ca-8e07-bab7e72e3c3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:48.704294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a3820-2819-49ca-8e07-bab7e72e3c3b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:48.721943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-804a3820-2819-49ca-8e07-bab7e72e3c3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:48.721943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:48.721943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:48.721943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:48.721943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:48.721943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:48.721943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:48.721943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:48.721943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:48.721943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:48.721943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:48.721943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:48.721943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:12:48.721943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:12:48.721943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:12:48.721943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:12:48.723477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:48.723477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:48.723477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:48.723477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:48.723477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:48.723477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:48.723477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:48.723477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:48.723477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:48.723477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:48.723477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:48.723477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:48.723477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:48.723477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:48.723477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:48.723477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:48.723477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:12:48.724583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:12:48.724583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:48.724583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:12:48.724583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 606/1224] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:48.729055 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101859e9-c74a-41e7-9757-bdb3c9e18f44 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:48.845839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101859e9-c74a-41e7-9757-bdb3c9e18f44 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:48.886212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41080b1b-e1d7-4048-9522-37927e9a1f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:48.887160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41080b1b-e1d7-4048-9522-37927e9a1f60 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:48.887516 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41080b1b-e1d7-4048-9522-37927e9a1f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:48.887908 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41080b1b-e1d7-4048-9522-37927e9a1f60 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:48.888684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41080b1b-e1d7-4048-9522-37927e9a1f60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:48.889088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41080b1b-e1d7-4048-9522-37927e9a1f60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:48.890580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41080b1b-e1d7-4048-9522-37927e9a1f60 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:48.903416 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 619/1225] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:12:48.909428 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a3820-2819-49ca-8e07-bab7e72e3c3b None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:49.018460 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:49.026705 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a3820-2819-49ca-8e07-bab7e72e3c3b None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:49.061249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396328e6-0692-4960-a5c4-a8a878e9aa1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:49.061885 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396328e6-0692-4960-a5c4-a8a878e9aa1b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:49.062165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396328e6-0692-4960-a5c4-a8a878e9aa1b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:49.062439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396328e6-0692-4960-a5c4-a8a878e9aa1b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:49.070046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396328e6-0692-4960-a5c4-a8a878e9aa1b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:49.071513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396328e6-0692-4960-a5c4-a8a878e9aa1b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:49.082934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-396328e6-0692-4960-a5c4-a8a878e9aa1b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 12:12:49.082934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:49.082934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:49.082934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:49.082934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:49.082934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:49.082934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:49.082934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:49.082934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:49.082934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:49.082934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:49.082934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:49.082934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:12:49.082934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:12:49.082934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:12:49.082934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:49.084121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:49.085518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:49.085518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:49.085518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:12:49.085518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:12:49.085518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:12:49.085518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:12:49.085518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:12:49.085518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:12:49.085518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 12:12:49.085518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:12:49.085518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 607/1226] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:12:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:12:49.090532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41080b1b-e1d7-4048-9522-37927e9a1f60 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:49.140071 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:49.200197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41080b1b-e1d7-4048-9522-37927e9a1f60 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:49.240145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbb971-8f59-4f51-8d8f-052559c95ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:49.240746 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbb971-8f59-4f51-8d8f-052559c95ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:49.241021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbb971-8f59-4f51-8d8f-052559c95ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:49.241318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbb971-8f59-4f51-8d8f-052559c95ff9 None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:49.242038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbb971-8f59-4f51-8d8f-052559c95ff9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:49.242347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbb971-8f59-4f51-8d8f-052559c95ff9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:49.243766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbb971-8f59-4f51-8d8f-052559c95ff9 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:49.262708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 620/1227] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Sep 28 12:12:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:12:49.269044 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396328e6-0692-4960-a5c4-a8a878e9aa1b None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:49.380420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396328e6-0692-4960-a5c4-a8a878e9aa1b None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:49.417687 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d42ea-dbca-48b1-8d0e-a70dd00d592e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:49.418405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d42ea-dbca-48b1-8d0e-a70dd00d592e None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:49.418696 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d42ea-dbca-48b1-8d0e-a70dd00d592e None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:49.419043 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d42ea-dbca-48b1-8d0e-a70dd00d592e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:49.430098 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d42ea-dbca-48b1-8d0e-a70dd00d592e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:49.431550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d42ea-dbca-48b1-8d0e-a70dd00d592e None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:49.447650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-374d42ea-dbca-48b1-8d0e-a70dd00d592e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:49.447650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:49.447650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:49.447650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:49.447650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:49.447650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:49.447650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:49.447650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:49.447650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:49.447650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:49.447650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:49.447650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:49.447650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:12:49.447650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:12:49.447650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:12:49.447650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:12:49.449259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:49.449259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:49.449259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:49.449259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:49.449259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:49.449259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:49.449259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:49.449259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:49.449259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:49.449259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:49.449259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:49.449259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:49.449259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:49.449259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:49.449259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:49.449259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:49.449259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:12:49.450276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:12:49.450276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:49.450276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:12:49.456429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 608/1228] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:12:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:49.456576 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbb971-8f59-4f51-8d8f-052559c95ff9 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:49.563209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbb971-8f59-4f51-8d8f-052559c95ff9 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:49.565494 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:49.598636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68bbf2-95e8-46a8-abba-c86d64479438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:49.599234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68bbf2-95e8-46a8-abba-c86d64479438 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:49.599508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68bbf2-95e8-46a8-abba-c86d64479438 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:49.599773 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a68bbf2-95e8-46a8-abba-c86d64479438 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:49.600398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68bbf2-95e8-46a8-abba-c86d64479438 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:49.600700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68bbf2-95e8-46a8-abba-c86d64479438 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:49.602068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a68bbf2-95e8-46a8-abba-c86d64479438 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:49.614071 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 621/1229] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:12:49.619736 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d42ea-dbca-48b1-8d0e-a70dd00d592e None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:49.629520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:49.735631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d42ea-dbca-48b1-8d0e-a70dd00d592e None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:49.776699 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b97dc-80cf-4ea4-971f-e50d1e33e15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:49.777367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b97dc-80cf-4ea4-971f-e50d1e33e15c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:49.777643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b97dc-80cf-4ea4-971f-e50d1e33e15c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:49.777912 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b97dc-80cf-4ea4-971f-e50d1e33e15c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:49.786216 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b97dc-80cf-4ea4-971f-e50d1e33e15c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:49.787726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b97dc-80cf-4ea4-971f-e50d1e33e15c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:49.801403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-338b97dc-80cf-4ea4-971f-e50d1e33e15c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:12:49.801403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:49.801403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:49.801403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:49.801403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:49.801403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:49.801403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:49.801403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:49.801403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:49.801403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:49.801403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:49.801403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:49.801403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:12:49.801403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:12:49.801403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:12:49.802513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:12:49.802513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:49.802513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:49.802513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:49.802513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:49.802513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:49.802513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:49.802513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:49.802513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:49.802513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:49.802513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:49.802513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:49.802513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:49.802513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:49.802513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:49.802513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:49.802513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:49.803571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:12:49.803571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:12:49.803571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:12:49.803571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:12:49.803571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:12:49.803571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:12:49.803571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:12:49.803571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:12:49.803571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 609/1230] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:12:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:49.808697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68bbf2-95e8-46a8-abba-c86d64479438 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:49.929073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a68bbf2-95e8-46a8-abba-c86d64479438 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:49.968038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af712e3e-9629-4dca-8fa6-92c6b40ce1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:49.968658 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af712e3e-9629-4dca-8fa6-92c6b40ce1ee None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:49.968972 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af712e3e-9629-4dca-8fa6-92c6b40ce1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:49.969261 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af712e3e-9629-4dca-8fa6-92c6b40ce1ee None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:49.969973 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af712e3e-9629-4dca-8fa6-92c6b40ce1ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:49.970294 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af712e3e-9629-4dca-8fa6-92c6b40ce1ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:49.971687 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af712e3e-9629-4dca-8fa6-92c6b40ce1ee None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:49.984673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 622/1231] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:12:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:12:49.991655 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b97dc-80cf-4ea4-971f-e50d1e33e15c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:50.061053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:50.098103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b97dc-80cf-4ea4-971f-e50d1e33e15c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:50.136762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a1d58-27a4-4fd8-9d28-d6fedf070332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:50.137552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a1d58-27a4-4fd8-9d28-d6fedf070332 None admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:50.137856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a1d58-27a4-4fd8-9d28-d6fedf070332 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:50.138134 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a1d58-27a4-4fd8-9d28-d6fedf070332 None admin] PATH_INFO: `/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/9a2322b1a76d40e49545f5677e247489/roles/60963384f3e54d9cbfbc8d12023a6cd9` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:50.153534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a1d58-27a4-4fd8-9d28-d6fedf070332 None admin] RBAC: Authorizing `identity:create_grant(project_id=6335d0db1f4844c39c255909189e0ce9, user_id=9a2322b1a76d40e49545f5677e247489, role_id=60963384f3e54d9cbfbc8d12023a6cd9)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:50.154966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a1d58-27a4-4fd8-9d28-d6fedf070332 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:50.169069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 610/1232] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:12:49 2022] PUT /identity/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/9a2322b1a76d40e49545f5677e247489/roles/60963384f3e54d9cbfbc8d12023a6cd9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:12:52.453772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e63103-48c3-4c56-a2d1-438b9dcb3496 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:52.454363 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e63103-48c3-4c56-a2d1-438b9dcb3496 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:52.454363 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e63103-48c3-4c56-a2d1-438b9dcb3496 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:52.455626 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 623/1233] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:12:52.460814 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6550c0-295d-4eea-8dfd-bfa42f1760f5 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:52.461224 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6550c0-295d-4eea-8dfd-bfa42f1760f5 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:52.461444 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6550c0-295d-4eea-8dfd-bfa42f1760f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:52.482755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-9e6550c0-295d-4eea-8dfd-bfa42f1760f5 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:12:52.483947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-9e6550c0-295d-4eea-8dfd-bfa42f1760f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:12:52.561178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 611/1234] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:12:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:12:52.827348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e63103-48c3-4c56-a2d1-438b9dcb3496 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:52.858345 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:52.951927 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e63103-48c3-4c56-a2d1-438b9dcb3496 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:52.989237 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4fcc4-f994-4572-a9a4-10161f71a990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:52.989877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4fcc4-f994-4572-a9a4-10161f71a990 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:52.990193 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4fcc4-f994-4572-a9a4-10161f71a990 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:52.990563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4fcc4-f994-4572-a9a4-10161f71a990 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:53.003079 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4fcc4-f994-4572-a9a4-10161f71a990 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:12:53.009092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-abf4fcc4-f994-4572-a9a4-10161f71a990 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:12:53.009092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:53.009092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:53.009092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:53.009092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:53.009092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:53.009092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:53.009092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:53.009092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:53.009092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:53.009092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:53.009092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:53.009092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:12:53.009092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:12:53.009092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:12:53.009092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:12:53.010067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:12:53.010067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:12:53.010067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:12:53.010067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:12:53.010067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:53.010067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:53.010067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:53.010067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:53.010067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:53.010067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:53.010067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:53.010067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:53.010067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:53.010067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:53.010067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:53.010067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:53.011487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:53.011487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:53.011487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:53.011487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:53.011487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:12:53.011487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:12:53.011487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:12:53.011487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:12:53.011487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:12:53.011487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:12:53.011487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 12:12:53.011487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:12:53.011487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 624/1235] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Sep 28 12:12:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:53.017547 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6550c0-295d-4eea-8dfd-bfa42f1760f5 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:53.075998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:53.141443 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6550c0-295d-4eea-8dfd-bfa42f1760f5 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:53.180455 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c23bec-42e3-4a55-9bb5-12a24ca9111a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:53.181189 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c23bec-42e3-4a55-9bb5-12a24ca9111a None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:53.181557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c23bec-42e3-4a55-9bb5-12a24ca9111a None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:53.181859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c23bec-42e3-4a55-9bb5-12a24ca9111a None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:53.183463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c23bec-42e3-4a55-9bb5-12a24ca9111a None admin] RBAC: Adding query filter params (name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:53.183951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c23bec-42e3-4a55-9bb5-12a24ca9111a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:53.186300 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c23bec-42e3-4a55-9bb5-12a24ca9111a None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:53.197558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 612/1236] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Sep 28 12:12:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:12:53.204646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4fcc4-f994-4572-a9a4-10161f71a990 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:53.301655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:53.321377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4fcc4-f994-4572-a9a4-10161f71a990 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:53.359904 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c1526-4609-4164-8e7b-830ccf52707d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:53.360884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c1526-4609-4164-8e7b-830ccf52707d None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:53.361195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c1526-4609-4164-8e7b-830ccf52707d None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:53.361472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c1526-4609-4164-8e7b-830ccf52707d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:53.372980 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c1526-4609-4164-8e7b-830ccf52707d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:53.374473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c1526-4609-4164-8e7b-830ccf52707d None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:53.391724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-960c1526-4609-4164-8e7b-830ccf52707d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:53.391724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:53.391724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:53.391724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:53.391724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:53.391724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:53.391724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:53.391724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:53.391724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:53.391724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:53.391724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:53.391724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:53.391724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:12:53.391724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:12:53.391724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:12:53.391724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:12:53.392844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:53.392844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:53.392844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:53.392844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:53.392844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:53.392844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:53.392844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:53.392844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:53.392844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:53.392844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:53.392844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:53.392844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:53.392844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:53.392844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:53.392844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:53.392844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:53.392844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:12:53.394023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:12:53.394023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:53.394023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:12:53.394023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 625/1237] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:12:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:53.399645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c23bec-42e3-4a55-9bb5-12a24ca9111a None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:53.510680 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c23bec-42e3-4a55-9bb5-12a24ca9111a None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:53.519563 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:53.548495 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add100a-8e78-421a-87a0-74fa4aef195b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:53.549123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add100a-8e78-421a-87a0-74fa4aef195b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:53.549434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add100a-8e78-421a-87a0-74fa4aef195b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:53.549698 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add100a-8e78-421a-87a0-74fa4aef195b None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:53.550341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add100a-8e78-421a-87a0-74fa4aef195b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:53.550638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add100a-8e78-421a-87a0-74fa4aef195b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:53.551998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add100a-8e78-421a-87a0-74fa4aef195b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:53.564288 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 613/1238] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:12:53.570098 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c1526-4609-4164-8e7b-830ccf52707d None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:53.678628 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c1526-4609-4164-8e7b-830ccf52707d None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:53.712156 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b82d45-0648-424c-bec6-cb0d804c3bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:53.712895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b82d45-0648-424c-bec6-cb0d804c3bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:53.713263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b82d45-0648-424c-bec6-cb0d804c3bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:53.714057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b82d45-0648-424c-bec6-cb0d804c3bf3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:53.716862 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:53.722834 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b82d45-0648-424c-bec6-cb0d804c3bf3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:53.724422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b82d45-0648-424c-bec6-cb0d804c3bf3 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:53.736795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-87b82d45-0648-424c-bec6-cb0d804c3bf3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 12:12:53.736795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:53.736795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:53.736795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:53.736795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:53.736795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:53.736795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:53.736795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:53.736795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:53.736795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:53.736795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:53.736795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:53.736795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:12:53.736795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:12:53.736795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:12:53.736795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:53.738045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:53.739371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:53.739371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:53.739371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:12:53.739371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:12:53.739371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:12:53.739371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:12:53.739371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:12:53.739371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:12:53.739371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 12:12:53.739371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:12:53.739371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 626/1239] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:12:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:53.743892 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add100a-8e78-421a-87a0-74fa4aef195b None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:53.856023 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add100a-8e78-421a-87a0-74fa4aef195b None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:53.896013 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc197e5d-9d9d-4610-be72-9adb27501113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:53.896684 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc197e5d-9d9d-4610-be72-9adb27501113 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:53.897028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc197e5d-9d9d-4610-be72-9adb27501113 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:53.897413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc197e5d-9d9d-4610-be72-9adb27501113 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:53.898207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc197e5d-9d9d-4610-be72-9adb27501113 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:53.898516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc197e5d-9d9d-4610-be72-9adb27501113 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:53.900114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc197e5d-9d9d-4610-be72-9adb27501113 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:53.921612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 614/1240] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Sep 28 12:12:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:12:53.927695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b82d45-0648-424c-bec6-cb0d804c3bf3 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:54.037851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b82d45-0648-424c-bec6-cb0d804c3bf3 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:54.075081 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5db95e-bc18-4082-a5fe-1534f6b281ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:54.075852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5db95e-bc18-4082-a5fe-1534f6b281ec None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:54.076222 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5db95e-bc18-4082-a5fe-1534f6b281ec None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:54.076635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5db95e-bc18-4082-a5fe-1534f6b281ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:54.145799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:54.154505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5db95e-bc18-4082-a5fe-1534f6b281ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:54.155948 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5db95e-bc18-4082-a5fe-1534f6b281ec None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:54.267062 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 627/1241] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:12:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 12:12:54.274307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc197e5d-9d9d-4610-be72-9adb27501113 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:54.303556 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:54.387380 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc197e5d-9d9d-4610-be72-9adb27501113 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:54.423696 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa839f5-d414-4ff5-8616-a30591be190b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:54.424354 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa839f5-d414-4ff5-8616-a30591be190b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:54.424645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa839f5-d414-4ff5-8616-a30591be190b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:54.424953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa839f5-d414-4ff5-8616-a30591be190b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:54.436920 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa839f5-d414-4ff5-8616-a30591be190b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:54.438674 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa839f5-d414-4ff5-8616-a30591be190b None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:54.455090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-eaa839f5-d414-4ff5-8616-a30591be190b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:54.455090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:54.455090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:54.455090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:54.455090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:54.455090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:54.455090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:54.455090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:54.455090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:54.455090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:54.455090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:54.455090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:54.455090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:12:54.455090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:12:54.455090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:12:54.455090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:12:54.456240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:54.456240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:54.456240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:54.456240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:54.456240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:54.456240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:54.456240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:54.456240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:54.456240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:54.456240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:54.456240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:54.456240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:54.456240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:54.456240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:54.456240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:54.456240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:54.456240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:12:54.457362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:12:54.457362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:54.457362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:12:54.457362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 615/1242] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:54.461994 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5db95e-bc18-4082-a5fe-1534f6b281ec None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:54.567130 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:54.574382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5db95e-bc18-4082-a5fe-1534f6b281ec None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:54.610806 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8530b-6646-4425-b3e8-52280410da93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:54.611417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8530b-6646-4425-b3e8-52280410da93 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:54.612047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8530b-6646-4425-b3e8-52280410da93 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:54.612047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8530b-6646-4425-b3e8-52280410da93 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:54.612640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8530b-6646-4425-b3e8-52280410da93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:54.612982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8530b-6646-4425-b3e8-52280410da93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:54.614622 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8530b-6646-4425-b3e8-52280410da93 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:54.627874 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 628/1243] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:12:54.633807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa839f5-d414-4ff5-8616-a30591be190b None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:54.725067 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:54.752148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa839f5-d414-4ff5-8616-a30591be190b None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:54.790731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7e087-32b8-4375-9692-2dbda6782b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:54.791444 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7e087-32b8-4375-9692-2dbda6782b56 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:54.792421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7e087-32b8-4375-9692-2dbda6782b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:54.792764 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7e087-32b8-4375-9692-2dbda6782b56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:54.800369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7e087-32b8-4375-9692-2dbda6782b56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:54.802154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7e087-32b8-4375-9692-2dbda6782b56 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:54.816651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-2ad7e087-32b8-4375-9692-2dbda6782b56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:12:54.816651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:54.816651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:54.816651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:54.816651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:54.816651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:54.816651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:54.816651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:54.816651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:54.816651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:54.816651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:54.816651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:54.816651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:12:54.816651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:12:54.816651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:12:54.817727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:12:54.817727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:54.817727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:54.817727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:54.817727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:54.817727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:54.817727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:54.817727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:54.817727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:54.817727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:54.817727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:54.817727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:54.817727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:54.817727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:54.817727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:54.817727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:54.817727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:54.823804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:12:54.823804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:12:54.823804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:12:54.823804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:12:54.823804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:12:54.823804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:12:54.823804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:12:54.823804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:12:54.823804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 616/1244] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:12:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:54.825534 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8530b-6646-4425-b3e8-52280410da93 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:54.948532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8530b-6646-4425-b3e8-52280410da93 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:54.986769 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20258e0-0703-44e2-9e9b-fea0ba8bdf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:54.987379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20258e0-0703-44e2-9e9b-fea0ba8bdf0a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:54.987664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20258e0-0703-44e2-9e9b-fea0ba8bdf0a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:54.987917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20258e0-0703-44e2-9e9b-fea0ba8bdf0a None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:54.988593 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20258e0-0703-44e2-9e9b-fea0ba8bdf0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:54.988897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20258e0-0703-44e2-9e9b-fea0ba8bdf0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:54.990359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20258e0-0703-44e2-9e9b-fea0ba8bdf0a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:55.002410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 629/1245] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:12:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:12:55.008646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7e087-32b8-4375-9692-2dbda6782b56 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:55.120649 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7e087-32b8-4375-9692-2dbda6782b56 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:55.159129 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77dcb5-5250-4d56-a4ec-18cca8e1c426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:55.159778 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77dcb5-5250-4d56-a4ec-18cca8e1c426 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:55.160058 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77dcb5-5250-4d56-a4ec-18cca8e1c426 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:55.160352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77dcb5-5250-4d56-a4ec-18cca8e1c426 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:55.161198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77dcb5-5250-4d56-a4ec-18cca8e1c426 None admin] RBAC: Adding query filter params (role.id=60963384f3e54d9cbfbc8d12023a6cd9, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=9a2322b1a76d40e49545f5677e247489) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:55.161515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77dcb5-5250-4d56-a4ec-18cca8e1c426 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:55.163081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77dcb5-5250-4d56-a4ec-18cca8e1c426 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:55.171882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 617/1246] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:12:55 2022] GET /identity/v3/role_assignments?user.id=9a2322b1a76d40e49545f5677e247489&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=60963384f3e54d9cbfbc8d12023a6cd9 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:12:57.594050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2558c3-6c45-4589-aad1-cc7a151d790b None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:57.594682 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2558c3-6c45-4589-aad1-cc7a151d790b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:57.594682 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2558c3-6c45-4589-aad1-cc7a151d790b None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:57.596112 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 630/1247] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:12:57.602796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f8c77-b2ba-466b-a2dc-6381ca737e51 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:57.603232 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f8c77-b2ba-466b-a2dc-6381ca737e51 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:57.603394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f8c77-b2ba-466b-a2dc-6381ca737e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:57.611189 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:57.626740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-809f8c77-b2ba-466b-a2dc-6381ca737e51 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:12:57.627945 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-809f8c77-b2ba-466b-a2dc-6381ca737e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:12:57.704245 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 618/1248] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:12:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:12:57.994247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2558c3-6c45-4589-aad1-cc7a151d790b None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:58.015401 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:58.110756 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2558c3-6c45-4589-aad1-cc7a151d790b None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:58.151966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e38f4-9064-423b-affa-cbc747d33103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:58.152711 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e38f4-9064-423b-affa-cbc747d33103 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:58.153020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e38f4-9064-423b-affa-cbc747d33103 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:58.153327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e38f4-9064-423b-affa-cbc747d33103 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:58.164776 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e38f4-9064-423b-affa-cbc747d33103 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:12:58.170211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-cb2e38f4-9064-423b-affa-cbc747d33103 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:12:58.170211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:58.170211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:58.170211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:58.170211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:58.170211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:58.170211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:58.170211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:58.170211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:58.170211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:58.170211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:58.170211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:58.170211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:12:58.170211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:12:58.170211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:12:58.170211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:12:58.171263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:12:58.171263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:12:58.171263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:12:58.171263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:12:58.171263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:58.171263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:58.171263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:58.171263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:58.171263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:58.171263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:58.171263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:58.171263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:58.171263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:58.171263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:58.171263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:58.171263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:58.172631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:58.172631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:58.172631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:58.172631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:58.172631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:12:58.172631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:12:58.172631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:12:58.172631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:12:58.172631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:12:58.172631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:12:58.172631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:12:58.172631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:12:58.172631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 631/1249] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:12:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:58.177945 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f8c77-b2ba-466b-a2dc-6381ca737e51 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:58.295793 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f8c77-b2ba-466b-a2dc-6381ca737e51 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:58.327544 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:58.332981 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808cdf95-e3d9-497e-857b-ee308df08f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:58.333676 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cdf95-e3d9-497e-857b-ee308df08f3d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:58.333963 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cdf95-e3d9-497e-857b-ee308df08f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:58.334270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cdf95-e3d9-497e-857b-ee308df08f3d None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:58.335000 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808cdf95-e3d9-497e-857b-ee308df08f3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:58.335361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808cdf95-e3d9-497e-857b-ee308df08f3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:58.336747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808cdf95-e3d9-497e-857b-ee308df08f3d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:58.351907 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 619/1250] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:12:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:12:58.353183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e38f4-9064-423b-affa-cbc747d33103 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:58.436938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:58.462664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e38f4-9064-423b-affa-cbc747d33103 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:58.499245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60d80d-315e-4b10-9f79-6e4f7b971a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:58.499933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60d80d-315e-4b10-9f79-6e4f7b971a5f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:58.500244 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60d80d-315e-4b10-9f79-6e4f7b971a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:58.500516 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60d80d-315e-4b10-9f79-6e4f7b971a5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:58.511937 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60d80d-315e-4b10-9f79-6e4f7b971a5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:58.513405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60d80d-315e-4b10-9f79-6e4f7b971a5f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:58.530455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-ce60d80d-315e-4b10-9f79-6e4f7b971a5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:58.530455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:58.530455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:58.530455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:58.530455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:58.530455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:58.530455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:58.530455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:58.530455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:58.530455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:58.530455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:58.530455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:58.530455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:12:58.530455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:12:58.530455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:12:58.530455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:12:58.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:58.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:58.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:58.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:58.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:58.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:58.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:58.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:58.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:58.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:58.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:58.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:58.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:58.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:58.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:58.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:58.531513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:12:58.532441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:12:58.532441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:58.532441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:12:58.532441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 632/1251] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:12:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 12:12:58.538236 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808cdf95-e3d9-497e-857b-ee308df08f3d None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:58.654389 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808cdf95-e3d9-497e-857b-ee308df08f3d None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:58.694788 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05810c26-5791-4148-99b4-a5d864897d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:58.695512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05810c26-5791-4148-99b4-a5d864897d2d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:58.695881 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05810c26-5791-4148-99b4-a5d864897d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:58.696218 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05810c26-5791-4148-99b4-a5d864897d2d None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:58.696866 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05810c26-5791-4148-99b4-a5d864897d2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:58.697260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05810c26-5791-4148-99b4-a5d864897d2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:58.698886 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05810c26-5791-4148-99b4-a5d864897d2d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:58.712361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 620/1252] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:12:58.719019 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60d80d-315e-4b10-9f79-6e4f7b971a5f None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:58.845711 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60d80d-315e-4b10-9f79-6e4f7b971a5f None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:58.880591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:58.890029 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5345a0e-247a-490a-b8e0-99ab19e966a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:58.890676 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5345a0e-247a-490a-b8e0-99ab19e966a0 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:58.891002 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5345a0e-247a-490a-b8e0-99ab19e966a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:58.891229 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5345a0e-247a-490a-b8e0-99ab19e966a0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:58.900450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5345a0e-247a-490a-b8e0-99ab19e966a0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:58.902549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5345a0e-247a-490a-b8e0-99ab19e966a0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:58.915354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-c5345a0e-247a-490a-b8e0-99ab19e966a0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 12:12:58.915354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:58.915354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:58.915354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:58.915354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:58.915354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:58.915354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:58.915354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:58.915354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:58.915354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:58.915354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:58.915354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:58.915354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:12:58.915354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:12:58.915354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:12:58.915354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:58.917255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:58.918466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:58.918466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:58.918466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:12:58.918466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:12:58.918466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:12:58.918466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:12:58.918466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:12:58.918466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:12:58.918466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 12:12:58.918466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:12:58.918466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 633/1253] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:12:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:58.923308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05810c26-5791-4148-99b4-a5d864897d2d None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:58.984337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:59.031161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05810c26-5791-4148-99b4-a5d864897d2d None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:59.067031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cebc0a-c07a-4fa3-a6e5-26af561fe278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:59.067648 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cebc0a-c07a-4fa3-a6e5-26af561fe278 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:59.067923 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cebc0a-c07a-4fa3-a6e5-26af561fe278 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:59.068191 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cebc0a-c07a-4fa3-a6e5-26af561fe278 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:59.068919 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cebc0a-c07a-4fa3-a6e5-26af561fe278 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:59.069258 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cebc0a-c07a-4fa3-a6e5-26af561fe278 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:59.070704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cebc0a-c07a-4fa3-a6e5-26af561fe278 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:59.090943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 621/1254] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Sep 28 12:12:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:12:59.096898 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5345a0e-247a-490a-b8e0-99ab19e966a0 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:59.206028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5345a0e-247a-490a-b8e0-99ab19e966a0 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:59.241733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e041b0-2280-4845-a850-5bfc3a569a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:59.242346 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e041b0-2280-4845-a850-5bfc3a569a90 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:59.242626 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e041b0-2280-4845-a850-5bfc3a569a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:59.242898 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e041b0-2280-4845-a850-5bfc3a569a90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:59.301438 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:59.315094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e041b0-2280-4845-a850-5bfc3a569a90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:59.316531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e041b0-2280-4845-a850-5bfc3a569a90 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:59.417525 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 634/1255] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:12:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:12:59.423526 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cebc0a-c07a-4fa3-a6e5-26af561fe278 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:59.524389 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cebc0a-c07a-4fa3-a6e5-26af561fe278 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:59.538354 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:59.560920 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2d7e4-83c2-45d3-b994-b9bff517904c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:59.561667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2d7e4-83c2-45d3-b994-b9bff517904c None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:59.561942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2d7e4-83c2-45d3-b994-b9bff517904c None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:59.562213 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2d7e4-83c2-45d3-b994-b9bff517904c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:59.572152 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af2d7e4-83c2-45d3-b994-b9bff517904c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:59.574308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af2d7e4-83c2-45d3-b994-b9bff517904c None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:59.588345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-4af2d7e4-83c2-45d3-b994-b9bff517904c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:59.588345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:59.588345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:59.588345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:59.588345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:59.588345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:59.588345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:59.588345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:59.588345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:59.588345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:59.588345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:59.588345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:59.588345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:12:59.588345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:12:59.588345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:12:59.588345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:12:59.589664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:59.589664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:59.589664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:59.589664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:59.589664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:59.589664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:59.589664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:59.589664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:59.589664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:59.589664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:59.589664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:59.589664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:59.589664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:59.589664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:59.589664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:59.589664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:59.589664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:12:59.590747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:12:59.590747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:12:59.590747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:12:59.590747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 622/1256] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:12:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:59.595403 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e041b0-2280-4845-a850-5bfc3a569a90 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:59.678854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:12:59.691067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e041b0-2280-4845-a850-5bfc3a569a90 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:59.723675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63786f-4712-4ed8-98c2-bc0391cd4d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:59.724309 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63786f-4712-4ed8-98c2-bc0391cd4d87 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:59.724526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63786f-4712-4ed8-98c2-bc0391cd4d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:59.724791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63786f-4712-4ed8-98c2-bc0391cd4d87 None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:59.725452 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63786f-4712-4ed8-98c2-bc0391cd4d87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:12:59.725750 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63786f-4712-4ed8-98c2-bc0391cd4d87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:59.727095 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63786f-4712-4ed8-98c2-bc0391cd4d87 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:59.738060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 635/1257] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:12:59.743674 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2d7e4-83c2-45d3-b994-b9bff517904c None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:12:59.849616 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2d7e4-83c2-45d3-b994-b9bff517904c None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:12:59.883244 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3abff-25a1-426a-8f2e-ea746693dfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:12:59.883859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3abff-25a1-426a-8f2e-ea746693dfdb None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:12:59.884136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3abff-25a1-426a-8f2e-ea746693dfdb None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:12:59.884407 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb3abff-25a1-426a-8f2e-ea746693dfdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:12:59.891502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3abff-25a1-426a-8f2e-ea746693dfdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:12:59.892918 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb3abff-25a1-426a-8f2e-ea746693dfdb None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:12:59.904704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-4eb3abff-25a1-426a-8f2e-ea746693dfdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:12:59.904704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:12:59.904704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:12:59.904704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:12:59.904704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:12:59.904704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:12:59.904704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:12:59.904704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:12:59.904704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:12:59.904704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:12:59.904704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:12:59.904704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:12:59.904704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:12:59.904704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:12:59.904704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:12:59.905796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:12:59.905796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:12:59.905796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:12:59.905796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:12:59.905796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:12:59.905796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:12:59.905796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:12:59.905796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:12:59.905796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:12:59.905796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:12:59.905796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:12:59.905796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:12:59.905796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:12:59.905796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:12:59.905796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:12:59.905796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:12:59.905796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:12:59.906799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:12:59.906799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:12:59.906799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:12:59.906799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:12:59.906799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:12:59.906799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:12:59.906799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:12:59.906799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:12:59.906799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 623/1258] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:12:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:12:59.911733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63786f-4712-4ed8-98c2-bc0391cd4d87 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:00.010324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63786f-4712-4ed8-98c2-bc0391cd4d87 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:00.042917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a0071-0512-405a-9c60-d5fb6e716523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:00.043513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a0071-0512-405a-9c60-d5fb6e716523 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:00.043792 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a0071-0512-405a-9c60-d5fb6e716523 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:00.044062 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a0071-0512-405a-9c60-d5fb6e716523 None admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:00.044738 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a0071-0512-405a-9c60-d5fb6e716523 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:00.045041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a0071-0512-405a-9c60-d5fb6e716523 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:00.046501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a0071-0512-405a-9c60-d5fb6e716523 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:00.058571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 636/1259] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:12:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:13:00.064866 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3abff-25a1-426a-8f2e-ea746693dfdb None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:00.089944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:00.162070 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb3abff-25a1-426a-8f2e-ea746693dfdb None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:00.195405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928243ec-719a-422f-86ae-673b70c0ff19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:00.196009 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928243ec-719a-422f-86ae-673b70c0ff19 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:00.196284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928243ec-719a-422f-86ae-673b70c0ff19 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:00.196551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928243ec-719a-422f-86ae-673b70c0ff19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:00.197297 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928243ec-719a-422f-86ae-673b70c0ff19 None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=9a2322b1a76d40e49545f5677e247489) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:00.197607 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928243ec-719a-422f-86ae-673b70c0ff19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:00.199074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928243ec-719a-422f-86ae-673b70c0ff19 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:00.204300 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 624/1260] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:13:00 2022] GET /identity/v3/role_assignments?user.id=9a2322b1a76d40e49545f5677e247489&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:13:02.325371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba7d47-fc07-49fd-a7e8-5248e1c5df15 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:02.325932 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba7d47-fc07-49fd-a7e8-5248e1c5df15 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:02.325932 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba7d47-fc07-49fd-a7e8-5248e1c5df15 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:02.327249 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 637/1261] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:13:02.332403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292a365-effb-419c-8744-474845c2b27b None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:02.332778 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292a365-effb-419c-8744-474845c2b27b None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:02.332962 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292a365-effb-419c-8744-474845c2b27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:02.351364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-7292a365-effb-419c-8744-474845c2b27b None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:13:02.352542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7292a365-effb-419c-8744-474845c2b27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:13:02.417615 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 625/1262] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:13:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:13:02.687298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba7d47-fc07-49fd-a7e8-5248e1c5df15 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:02.698978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:02.786819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba7d47-fc07-49fd-a7e8-5248e1c5df15 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:02.823331 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b602552-4c30-4d3f-8b89-8a51dc1ec19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:02.824107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b602552-4c30-4d3f-8b89-8a51dc1ec19e None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:02.824481 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b602552-4c30-4d3f-8b89-8a51dc1ec19e None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:02.824817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b602552-4c30-4d3f-8b89-8a51dc1ec19e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:02.838995 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b602552-4c30-4d3f-8b89-8a51dc1ec19e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:13:02.845795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-9b602552-4c30-4d3f-8b89-8a51dc1ec19e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:13:02.845795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:13:02.845795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:13:02.845795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:13:02.845795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:13:02.845795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:13:02.845795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:13:02.845795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:13:02.845795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:13:02.845795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:13:02.845795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:13:02.845795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:13:02.845795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:13:02.845795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:13:02.845795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:13:02.845795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:13:02.846903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:13:02.846903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:13:02.846903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:13:02.846903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:13:02.846903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:13:02.846903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:13:02.846903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:13:02.846903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:13:02.846903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:13:02.846903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:13:02.846903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:13:02.846903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:13:02.846903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:13:02.846903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:13:02.846903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:13:02.846903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:13:02.848018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:13:02.848018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:13:02.848018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:13:02.848018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:13:02.848018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:13:02.848018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:13:02.848018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:13:02.848018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:13:02.848018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:13:02.848018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:13:02.848018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:13:02.848018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:13:02.848018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 638/1263] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:13:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:13:02.854618 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292a365-effb-419c-8744-474845c2b27b None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:02.879829 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:02.960211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292a365-effb-419c-8744-474845c2b27b None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:02.993042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57a3c8-fcbc-4318-bc02-1cea8423dcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:02.993668 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57a3c8-fcbc-4318-bc02-1cea8423dcba None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:02.993949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57a3c8-fcbc-4318-bc02-1cea8423dcba None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:02.994221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57a3c8-fcbc-4318-bc02-1cea8423dcba None admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:02.995624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57a3c8-fcbc-4318-bc02-1cea8423dcba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:02.995926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57a3c8-fcbc-4318-bc02-1cea8423dcba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:02.997320 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57a3c8-fcbc-4318-bc02-1cea8423dcba None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:03.005121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 626/1264] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:13:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:13:03.011047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b602552-4c30-4d3f-8b89-8a51dc1ec19e None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:03.075867 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:03.108139 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b602552-4c30-4d3f-8b89-8a51dc1ec19e None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:03.141534 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4cee5-7ddb-4c49-949c-944d498c6d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:03.142150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4cee5-7ddb-4c49-949c-944d498c6d14 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:03.142426 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4cee5-7ddb-4c49-949c-944d498c6d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:03.142692 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4cee5-7ddb-4c49-949c-944d498c6d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:03.152377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4cee5-7ddb-4c49-949c-944d498c6d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:03.153844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd4cee5-7ddb-4c49-949c-944d498c6d14 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:03.168245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-7cd4cee5-7ddb-4c49-949c-944d498c6d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:13:03.168245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:13:03.168245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:13:03.168245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:13:03.168245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:13:03.168245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:13:03.168245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:13:03.168245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:13:03.168245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:13:03.168245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:13:03.168245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:13:03.168245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:13:03.168245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:13:03.168245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:13:03.168245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:13:03.168245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:13:03.169280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:13:03.169280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:13:03.169280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:13:03.169280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:13:03.169280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:13:03.169280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:13:03.169280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:13:03.169280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:13:03.169280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:13:03.169280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:13:03.169280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:13:03.169280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:13:03.169280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:13:03.169280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:13:03.169280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:13:03.169280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:13:03.169280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:13:03.170212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:13:03.170212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:13:03.170212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:13:03.170212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 639/1265] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:13:03.175069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57a3c8-fcbc-4318-bc02-1cea8423dcba None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:03.254659 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:03.273427 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57a3c8-fcbc-4318-bc02-1cea8423dcba None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:03.306636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b044ab-bd09-45aa-a3e8-a50b0e9d1529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:03.307275 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b044ab-bd09-45aa-a3e8-a50b0e9d1529 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:03.307545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b044ab-bd09-45aa-a3e8-a50b0e9d1529 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:03.307813 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b044ab-bd09-45aa-a3e8-a50b0e9d1529 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:03.308447 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b044ab-bd09-45aa-a3e8-a50b0e9d1529 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:03.308746 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b044ab-bd09-45aa-a3e8-a50b0e9d1529 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:03.310124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b044ab-bd09-45aa-a3e8-a50b0e9d1529 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:03.321388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 627/1266] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:13:03.327123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4cee5-7ddb-4c49-949c-944d498c6d14 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:03.426965 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd4cee5-7ddb-4c49-949c-944d498c6d14 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:03.446712 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:03.461385 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b4320-370c-47da-8432-212e890fa36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:03.462007 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b4320-370c-47da-8432-212e890fa36b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:03.462288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b4320-370c-47da-8432-212e890fa36b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:03.462558 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b4320-370c-47da-8432-212e890fa36b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:03.469977 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b4320-370c-47da-8432-212e890fa36b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:03.471391 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b4320-370c-47da-8432-212e890fa36b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:03.482791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-ea0b4320-370c-47da-8432-212e890fa36b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 12:13:03.482791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:13:03.482791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:13:03.482791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:13:03.482791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:13:03.482791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:13:03.482791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:13:03.482791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:13:03.482791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:13:03.482791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:13:03.482791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:13:03.482791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:13:03.482791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:13:03.482791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:13:03.482791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:13:03.482791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:13:03.483905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:13:03.485316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:13:03.485316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:13:03.485316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:13:03.485316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:13:03.485316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:13:03.485316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:13:03.485316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:13:03.485316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:13:03.485316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 12:13:03.485316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:13:03.485316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 640/1267] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:13:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:13:03.489965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b044ab-bd09-45aa-a3e8-a50b0e9d1529 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:03.586966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b044ab-bd09-45aa-a3e8-a50b0e9d1529 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:03.619967 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1193671-18f0-4761-9d1a-0a082325c460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:03.620560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1193671-18f0-4761-9d1a-0a082325c460 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:03.620837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1193671-18f0-4761-9d1a-0a082325c460 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:03.621102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1193671-18f0-4761-9d1a-0a082325c460 None admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:03.621847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1193671-18f0-4761-9d1a-0a082325c460 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:03.622147 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1193671-18f0-4761-9d1a-0a082325c460 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:03.623555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1193671-18f0-4761-9d1a-0a082325c460 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:03.631866 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:03.642651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 628/1268] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Sep 28 12:13:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:13:03.648347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b4320-370c-47da-8432-212e890fa36b None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:03.744336 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b4320-370c-47da-8432-212e890fa36b None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:03.776432 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195d932-2026-4e02-9629-8322ded3b61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:03.777044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195d932-2026-4e02-9629-8322ded3b61f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:03.777348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195d932-2026-4e02-9629-8322ded3b61f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:03.777618 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195d932-2026-4e02-9629-8322ded3b61f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:03.787509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195d932-2026-4e02-9629-8322ded3b61f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:03.788922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195d932-2026-4e02-9629-8322ded3b61f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:03.802884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-9195d932-2026-4e02-9629-8322ded3b61f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:13:03.802884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:13:03.802884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:13:03.802884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:13:03.802884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:13:03.802884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:13:03.802884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:13:03.802884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:13:03.802884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:13:03.802884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:13:03.802884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:13:03.802884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:13:03.802884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:13:03.802884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:13:03.802884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:13:03.802884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:13:03.803983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:13:03.803983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:13:03.803983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:13:03.803983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:13:03.803983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:13:03.803983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:13:03.803983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:13:03.803983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:13:03.803983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:13:03.803983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:13:03.803983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:13:03.803983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:13:03.803983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:13:03.803983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:13:03.803983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:13:03.803983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:13:03.803983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:13:03.805325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:13:03.805325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:13:03.805325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:13:03.805325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 641/1269] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:13:03.810073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1193671-18f0-4761-9d1a-0a082325c460 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:03.909508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1193671-18f0-4761-9d1a-0a082325c460 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:03.942448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690edc7-ad7b-44c4-b803-966594ca76b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:03.943042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690edc7-ad7b-44c4-b803-966594ca76b8 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:03.943324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690edc7-ad7b-44c4-b803-966594ca76b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:03.943593 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690edc7-ad7b-44c4-b803-966594ca76b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:03.944227 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690edc7-ad7b-44c4-b803-966594ca76b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:03.944530 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690edc7-ad7b-44c4-b803-966594ca76b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:03.945908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690edc7-ad7b-44c4-b803-966594ca76b8 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:03.956801 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 629/1270] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:13:03.962477 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195d932-2026-4e02-9629-8322ded3b61f None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:03.993539 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:04.060700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195d932-2026-4e02-9629-8322ded3b61f None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:04.093413 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd7fbc-be2a-4b1e-aa24-5f2c0d28ea37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:04.094036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd7fbc-be2a-4b1e-aa24-5f2c0d28ea37 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:04.094315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd7fbc-be2a-4b1e-aa24-5f2c0d28ea37 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:04.094591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd7fbc-be2a-4b1e-aa24-5f2c0d28ea37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:04.100935 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd7fbc-be2a-4b1e-aa24-5f2c0d28ea37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:04.102398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd7fbc-be2a-4b1e-aa24-5f2c0d28ea37 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:04.114186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-03fd7fbc-be2a-4b1e-aa24-5f2c0d28ea37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:13:04.114186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:13:04.114186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:13:04.114186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:13:04.114186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:13:04.114186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:13:04.114186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:13:04.114186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:13:04.114186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:13:04.114186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:13:04.114186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:13:04.114186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:13:04.114186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:13:04.114186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:13:04.114186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:13:04.115216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:13:04.115216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:13:04.115216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:13:04.115216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:13:04.115216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:13:04.115216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:13:04.115216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:13:04.115216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:13:04.115216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:13:04.115216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:13:04.115216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:13:04.115216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:13:04.115216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:13:04.115216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:13:04.115216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:13:04.115216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:13:04.115216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:13:04.116307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:13:04.116307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:13:04.116307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:13:04.116307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:13:04.116307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:13:04.116307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:13:04.116307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:13:04.116307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:13:04.116307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 642/1271] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:13:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:13:04.121258 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690edc7-ad7b-44c4-b803-966594ca76b8 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:04.180177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:04.221495 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690edc7-ad7b-44c4-b803-966594ca76b8 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:04.254461 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6983e5-6a53-4350-8cca-fb46d9dff1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:04.255059 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6983e5-6a53-4350-8cca-fb46d9dff1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:04.255339 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6983e5-6a53-4350-8cca-fb46d9dff1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:04.255609 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6983e5-6a53-4350-8cca-fb46d9dff1d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:04.256291 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6983e5-6a53-4350-8cca-fb46d9dff1d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:04.256596 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6983e5-6a53-4350-8cca-fb46d9dff1d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:04.258043 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6983e5-6a53-4350-8cca-fb46d9dff1d6 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:04.269726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 630/1272] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:13:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:13:04.276136 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd7fbc-be2a-4b1e-aa24-5f2c0d28ea37 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:04.364524 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:04.373349 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd7fbc-be2a-4b1e-aa24-5f2c0d28ea37 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:04.405635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8c156-be70-4c9c-b3ba-b5c058857ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:04.406287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8c156-be70-4c9c-b3ba-b5c058857ea6 None admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:04.406568 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8c156-be70-4c9c-b3ba-b5c058857ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:04.406841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8c156-be70-4c9c-b3ba-b5c058857ea6 None admin] PATH_INFO: `/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/9a2322b1a76d40e49545f5677e247489/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:04.408692 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8c156-be70-4c9c-b3ba-b5c058857ea6 None admin] RBAC: Authorizing `identity:create_grant(project_id=6335d0db1f4844c39c255909189e0ce9, user_id=9a2322b1a76d40e49545f5677e247489, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:04.410141 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc8c156-be70-4c9c-b3ba-b5c058857ea6 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:04.425217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 643/1273] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Sep 28 12:13:04 2022] PUT /identity/v3/projects/6335d0db1f4844c39c255909189e0ce9/users/9a2322b1a76d40e49545f5677e247489/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:13:06.545898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001659d-8514-44fd-9ec2-4b869d3e9472 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:06.546427 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001659d-8514-44fd-9ec2-4b869d3e9472 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:06.546427 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001659d-8514-44fd-9ec2-4b869d3e9472 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:06.547764 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 631/1274] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:13:06.552916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29466009-00d9-43be-b584-cd163122e894 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:06.553337 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29466009-00d9-43be-b584-cd163122e894 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:06.553511 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29466009-00d9-43be-b584-cd163122e894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:06.571310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-29466009-00d9-43be-b584-cd163122e894 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:13:06.572570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-29466009-00d9-43be-b584-cd163122e894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:13:06.637617 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 644/1275] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:13:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:13:06.906865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001659d-8514-44fd-9ec2-4b869d3e9472 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:07.005440 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001659d-8514-44fd-9ec2-4b869d3e9472 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:07.022580 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:07.039741 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aea2b-ae6f-417d-80b5-e8874191234b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:07.040359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aea2b-ae6f-417d-80b5-e8874191234b None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:07.040637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aea2b-ae6f-417d-80b5-e8874191234b None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:07.040905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aea2b-ae6f-417d-80b5-e8874191234b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:07.050659 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392aea2b-ae6f-417d-80b5-e8874191234b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 12:13:07.055419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-392aea2b-ae6f-417d-80b5-e8874191234b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:13:07.055419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:13:07.055419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:13:07.055419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:13:07.055419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:13:07.055419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:13:07.055419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:13:07.055419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:13:07.055419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:13:07.055419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:13:07.055419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:13:07.055419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:13:07.055419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 12:13:07.055419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 12:13:07.055419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 12:13:07.055419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 12:13:07.056512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 12:13:07.056512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 12:13:07.056512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 12:13:07.056512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 12:13:07.056512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:13:07.056512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:13:07.056512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:13:07.056512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:13:07.056512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:13:07.056512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:13:07.056512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:13:07.056512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:13:07.056512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:13:07.056512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:13:07.056512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:13:07.056512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:13:07.058016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:13:07.058016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:13:07.058016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:13:07.058016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:13:07.058016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 12:13:07.058016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 12:13:07.058016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 12:13:07.058016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 12:13:07.058016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 12:13:07.058016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 12:13:07.058016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 12:13:07.058016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:13:07.058016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 632/1276] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Sep 28 12:13:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:13:07.063134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29466009-00d9-43be-b584-cd163122e894 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:07.140812 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:07.158764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29466009-00d9-43be-b584-cd163122e894 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:07.191193 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17057ee6-91da-4742-b101-f86bc5a954b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:07.191790 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17057ee6-91da-4742-b101-f86bc5a954b3 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:07.192064 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17057ee6-91da-4742-b101-f86bc5a954b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:07.192331 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17057ee6-91da-4742-b101-f86bc5a954b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:07.192976 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17057ee6-91da-4742-b101-f86bc5a954b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:07.193308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17057ee6-91da-4742-b101-f86bc5a954b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:07.194653 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17057ee6-91da-4742-b101-f86bc5a954b3 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:07.203092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 645/1277] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Sep 28 12:13:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:13:07.208865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aea2b-ae6f-417d-80b5-e8874191234b None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:07.306442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aea2b-ae6f-417d-80b5-e8874191234b None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:07.340512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c138bd8-8c78-4739-8f37-8043b5b48754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:07.341121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c138bd8-8c78-4739-8f37-8043b5b48754 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:07.341432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c138bd8-8c78-4739-8f37-8043b5b48754 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:07.341706 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c138bd8-8c78-4739-8f37-8043b5b48754 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:07.351425 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c138bd8-8c78-4739-8f37-8043b5b48754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:07.352863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c138bd8-8c78-4739-8f37-8043b5b48754 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:07.367597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-1c138bd8-8c78-4739-8f37-8043b5b48754 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:13:07.367597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:13:07.367597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:13:07.367597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:13:07.367597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:13:07.367597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:13:07.367597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:13:07.367597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:13:07.367597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:13:07.367597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:13:07.367597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:13:07.367597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:13:07.367597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:13:07.367597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:13:07.367597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:13:07.367597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:13:07.368769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:13:07.368769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:13:07.368769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:13:07.368769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:13:07.368769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:13:07.368769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:13:07.368769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:13:07.368769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:13:07.368769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:13:07.368769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:13:07.368769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:13:07.368769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:13:07.368769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:13:07.368769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:13:07.368769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:13:07.368769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:13:07.368769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:13:07.370428 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:13:07.370428 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:13:07.370428 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:13:07.370428 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 633/1278] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:13:07.374917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17057ee6-91da-4742-b101-f86bc5a954b3 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:07.469386 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17057ee6-91da-4742-b101-f86bc5a954b3 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:07.502163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88200d1-fa6b-4f6e-9e0f-a7521dd956cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:07.502748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88200d1-fa6b-4f6e-9e0f-a7521dd956cc None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:07.503020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88200d1-fa6b-4f6e-9e0f-a7521dd956cc None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:07.503282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88200d1-fa6b-4f6e-9e0f-a7521dd956cc None admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:07.503898 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88200d1-fa6b-4f6e-9e0f-a7521dd956cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:07.504232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88200d1-fa6b-4f6e-9e0f-a7521dd956cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:07.505560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88200d1-fa6b-4f6e-9e0f-a7521dd956cc None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:07.516911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 646/1279] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:13:07.522353 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c138bd8-8c78-4739-8f37-8043b5b48754 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:07.553467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:07.621391 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c138bd8-8c78-4739-8f37-8043b5b48754 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:07.653179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15285e67-8ca4-487e-99b2-559f3942a44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:07.653801 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15285e67-8ca4-487e-99b2-559f3942a44d None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:07.654083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15285e67-8ca4-487e-99b2-559f3942a44d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:07.654356 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15285e67-8ca4-487e-99b2-559f3942a44d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:07.661446 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15285e67-8ca4-487e-99b2-559f3942a44d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:07.662863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15285e67-8ca4-487e-99b2-559f3942a44d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:07.674355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-15285e67-8ca4-487e-99b2-559f3942a44d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 12:13:07.674355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:13:07.674355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:13:07.674355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:13:07.674355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:13:07.674355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:13:07.674355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:13:07.674355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:13:07.674355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:13:07.674355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:13:07.674355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:13:07.674355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:13:07.674355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 12:13:07.674355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 12:13:07.674355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 12:13:07.674355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:13:07.675345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:13:07.676418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:13:07.676418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:13:07.676418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 12:13:07.676418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 12:13:07.676418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 12:13:07.676418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 12:13:07.676418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 12:13:07.676418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 12:13:07.676418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 12:13:07.676418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:13:07.676418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 634/1280] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:13:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:13:07.681088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88200d1-fa6b-4f6e-9e0f-a7521dd956cc None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:07.685428 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:07.777557 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88200d1-fa6b-4f6e-9e0f-a7521dd956cc None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:07.809969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f38e0-eb9f-42dc-b9e0-5220a384566a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:07.810571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f38e0-eb9f-42dc-b9e0-5220a384566a None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:07.810860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f38e0-eb9f-42dc-b9e0-5220a384566a None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:07.811384 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f38e0-eb9f-42dc-b9e0-5220a384566a None admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:07.811977 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f38e0-eb9f-42dc-b9e0-5220a384566a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:07.812325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f38e0-eb9f-42dc-b9e0-5220a384566a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:07.814037 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f38e0-eb9f-42dc-b9e0-5220a384566a None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:07.831684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 647/1281] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Sep 28 12:13:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:13:07.837387 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15285e67-8ca4-487e-99b2-559f3942a44d None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:07.925854 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:07.935141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15285e67-8ca4-487e-99b2-559f3942a44d None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:07.968008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1e786-fecd-4d10-b9bf-227b1f7a6751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:07.968601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1e786-fecd-4d10-b9bf-227b1f7a6751 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:07.968882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1e786-fecd-4d10-b9bf-227b1f7a6751 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:07.969169 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1e786-fecd-4d10-b9bf-227b1f7a6751 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:08.034238 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1e786-fecd-4d10-b9bf-227b1f7a6751 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:08.035608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1e786-fecd-4d10-b9bf-227b1f7a6751 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:08.119729 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:08.129948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 635/1282] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Sep 28 12:13:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 12:13:08.135904 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f38e0-eb9f-42dc-b9e0-5220a384566a None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:08.195226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:08.232767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f38e0-eb9f-42dc-b9e0-5220a384566a None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:08.265658 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912c2b0-11ba-4894-99b3-0127fd9fbc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:08.266307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912c2b0-11ba-4894-99b3-0127fd9fbc00 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:08.266584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912c2b0-11ba-4894-99b3-0127fd9fbc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:08.266854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912c2b0-11ba-4894-99b3-0127fd9fbc00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:08.277027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912c2b0-11ba-4894-99b3-0127fd9fbc00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:08.278476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912c2b0-11ba-4894-99b3-0127fd9fbc00 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:08.292501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-b912c2b0-11ba-4894-99b3-0127fd9fbc00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:13:08.292501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:13:08.292501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:13:08.292501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:13:08.292501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:13:08.292501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:13:08.292501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:13:08.292501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:13:08.292501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:13:08.292501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:13:08.292501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:13:08.292501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:13:08.292501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 12:13:08.292501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 12:13:08.292501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 12:13:08.292501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 12:13:08.293633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:13:08.293633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:13:08.293633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:13:08.293633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:13:08.293633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:13:08.293633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:13:08.293633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:13:08.293633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:13:08.293633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:13:08.293633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:13:08.293633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:13:08.293633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:13:08.293633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:13:08.293633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:13:08.293633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:13:08.293633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:13:08.293633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 12:13:08.294549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 12:13:08.294549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 12:13:08.294549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:13:08.294549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 648/1283] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:13:08.299324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1e786-fecd-4d10-b9bf-227b1f7a6751 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:08.394313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1e786-fecd-4d10-b9bf-227b1f7a6751 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:08.426970 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2712479-3586-43a6-b765-160c32a537f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:08.427558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2712479-3586-43a6-b765-160c32a537f8 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:08.427836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2712479-3586-43a6-b765-160c32a537f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:08.428106 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2712479-3586-43a6-b765-160c32a537f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:08.428735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2712479-3586-43a6-b765-160c32a537f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:08.429033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2712479-3586-43a6-b765-160c32a537f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:08.430418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2712479-3586-43a6-b765-160c32a537f8 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:08.441352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 636/1284] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Sep 28 12:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:13:08.446931 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912c2b0-11ba-4894-99b3-0127fd9fbc00 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:08.544719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912c2b0-11ba-4894-99b3-0127fd9fbc00 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:08.558107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:08.578606 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931c9c4-aabb-43e8-a960-4e2ba53733a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:08.579219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931c9c4-aabb-43e8-a960-4e2ba53733a6 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:08.579496 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931c9c4-aabb-43e8-a960-4e2ba53733a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:08.579764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931c9c4-aabb-43e8-a960-4e2ba53733a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:08.586157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931c9c4-aabb-43e8-a960-4e2ba53733a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:08.587572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931c9c4-aabb-43e8-a960-4e2ba53733a6 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:08.599514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application [None req-1931c9c4-aabb-43e8-a960-4e2ba53733a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:13:08.599514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:13:08.599514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:13:08.599514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:13:08.599514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:13:08.599514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:13:08.599514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:13:08.599514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:13:08.599514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:13:08.599514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:13:08.599514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:13:08.599514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:13:08.599514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 12:13:08.599514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 12:13:08.599514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 12:13:08.600581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 12:13:08.600581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:13:08.600581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:13:08.600581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:13:08.600581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:13:08.600581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:13:08.600581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:13:08.600581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:13:08.600581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application with Lock( Sep 28 12:13:08.600581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:13:08.600581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:13:08.600581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:13:08.600581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:13:08.600581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:13:08.600581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:13:08.600581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:13:08.600581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:13:08.601547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 12:13:08.601547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 12:13:08.601547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 12:13:08.601547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 12:13:08.601547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 12:13:08.601547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 12:13:08.601547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 12:13:08.601547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: ERROR keystone.server.flask.application Sep 28 12:13:08.601547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 649/1285] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Sep 28 12:13:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:13:08.606574 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2712479-3586-43a6-b765-160c32a537f8 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:08.656065 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:08.705039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2712479-3586-43a6-b765-160c32a537f8 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:08.737189 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a79998-1e56-4605-8555-2e3e6ef558d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:08.737785 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a79998-1e56-4605-8555-2e3e6ef558d1 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:08.738066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a79998-1e56-4605-8555-2e3e6ef558d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:08.738343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a79998-1e56-4605-8555-2e3e6ef558d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:08.739028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a79998-1e56-4605-8555-2e3e6ef558d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:08.739334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a79998-1e56-4605-8555-2e3e6ef558d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:08.740751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a79998-1e56-4605-8555-2e3e6ef558d1 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:08.753564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 637/1286] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Sep 28 12:13:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:13:08.759735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931c9c4-aabb-43e8-a960-4e2ba53733a6 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:08.861508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931c9c4-aabb-43e8-a960-4e2ba53733a6 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:08.894613 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8850a594-ea3a-4763-8de0-0304e827282b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:08.895205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850a594-ea3a-4763-8de0-0304e827282b None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:08.895483 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850a594-ea3a-4763-8de0-0304e827282b None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:08.895752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8850a594-ea3a-4763-8de0-0304e827282b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:08.896462 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850a594-ea3a-4763-8de0-0304e827282b None admin] RBAC: Adding query filter params (role.id=c11f371c53dd40aab2cdc96e05805715, scope.project.id=6335d0db1f4844c39c255909189e0ce9, user.id=9a2322b1a76d40e49545f5677e247489) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:08.896766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850a594-ea3a-4763-8de0-0304e827282b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:08.898205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8850a594-ea3a-4763-8de0-0304e827282b None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:08.903818 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 650/1287] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Sep 28 12:13:08 2022] GET /identity/v3/role_assignments?user.id=9a2322b1a76d40e49545f5677e247489&scope.project.id=6335d0db1f4844c39c255909189e0ce9&role.id=c11f371c53dd40aab2cdc96e05805715 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:13:11.052453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc24df2-19c3-40ae-8d7d-bf4dbcc92b8a None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:11.053067 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc24df2-19c3-40ae-8d7d-bf4dbcc92b8a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:11.053182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc24df2-19c3-40ae-8d7d-bf4dbcc92b8a None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:11.054746 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 638/1288] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:13:11.060547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd439c-9c15-42b3-96bd-603339722dfe None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:11.060931 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd439c-9c15-42b3-96bd-603339722dfe None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:11.061094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd439c-9c15-42b3-96bd-603339722dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:11.079356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7fcd439c-9c15-42b3-96bd-603339722dfe None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:13:11.080521 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7fcd439c-9c15-42b3-96bd-603339722dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:13:11.109422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:11.147416 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 651/1289] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:13:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:13:11.411533 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc24df2-19c3-40ae-8d7d-bf4dbcc92b8a None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:11.510710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc24df2-19c3-40ae-8d7d-bf4dbcc92b8a None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:11.520939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:11.549408 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7279301-61f7-4386-9f15-9bbc9b4fe034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:11.550024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7279301-61f7-4386-9f15-9bbc9b4fe034 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:11.550302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7279301-61f7-4386-9f15-9bbc9b4fe034 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:11.550571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7279301-61f7-4386-9f15-9bbc9b4fe034 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:11.551138 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7279301-61f7-4386-9f15-9bbc9b4fe034 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:11.552487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7279301-61f7-4386-9f15-9bbc9b4fe034 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:11.561042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-b7279301-61f7-4386-9f15-9bbc9b4fe034 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 12:13:11.561042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:13:11.561042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:13:11.561042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:13:11.561042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:13:11.561042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:13:11.561042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:13:11.561042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:13:11.561042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 12:13:11.561042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:13:11.561042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:13:11.561042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:13:11.561042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:13:11.561042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:13:11.561042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:13:11.561042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:13:11.561042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:13:11.562249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:13:11.562249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:13:11.562249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:13:11.562249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:13:11.562249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:13:11.562249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:13:11.562249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:13:11.562249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:13:11.562249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:13:11.562249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:13:11.562249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:13:11.562249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:13:11.562249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:13:11.562249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:13:11.562249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:13:11.562249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:13:11.563100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:13:11.563100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:13:11.563100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:13:11.563100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:13:11.563100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:13:11.563100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:13:11.563100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:13:11.563100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:13:11.563100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:13:11.563100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:13:11.563100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:13:11.563100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:13:11.563100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 12:13:11.563100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:13:11.563100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 639/1290] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 12:13:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:13:11.568218 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd439c-9c15-42b3-96bd-603339722dfe None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:11.662463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd439c-9c15-42b3-96bd-603339722dfe None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:11.694920 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203c79a0-7c2f-4f57-baee-a1bdd6870e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:11.695510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c79a0-7c2f-4f57-baee-a1bdd6870e6f None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:11.695786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c79a0-7c2f-4f57-baee-a1bdd6870e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:11.696056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c79a0-7c2f-4f57-baee-a1bdd6870e6f None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:11.696670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203c79a0-7c2f-4f57-baee-a1bdd6870e6f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:11.696969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203c79a0-7c2f-4f57-baee-a1bdd6870e6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:11.698394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203c79a0-7c2f-4f57-baee-a1bdd6870e6f None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:11.702975 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 652/1291] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Sep 28 12:13:11 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:13:11.708553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7279301-61f7-4386-9f15-9bbc9b4fe034 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:11.804951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7279301-61f7-4386-9f15-9bbc9b4fe034 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:11.845817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c801ec3-e70c-448e-9116-b9295580aa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:11.846690 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c801ec3-e70c-448e-9116-b9295580aa43 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:11.846975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c801ec3-e70c-448e-9116-b9295580aa43 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:11.847298 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c801ec3-e70c-448e-9116-b9295580aa43 None admin] PATH_INFO: `/v3/services` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:11.848123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c801ec3-e70c-448e-9116-b9295580aa43 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:11.848433 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c801ec3-e70c-448e-9116-b9295580aa43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:11.850309 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c801ec3-e70c-448e-9116-b9295580aa43 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:11.856773 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 640/1292] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Sep 28 12:13:11 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:13:13.982774 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba322c5-4603-43ab-b95b-cc10585f2610 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:13.983309 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba322c5-4603-43ab-b95b-cc10585f2610 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:13.983309 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba322c5-4603-43ab-b95b-cc10585f2610 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:13.984630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 653/1293] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:13:13.990044 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd0e1c-4c53-434e-9127-4121fff36b68 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:13.990413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd0e1c-4c53-434e-9127-4121fff36b68 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:13.990602 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd0e1c-4c53-434e-9127-4121fff36b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:14.008407 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-33dd0e1c-4c53-434e-9127-4121fff36b68 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:13:14.009688 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-33dd0e1c-4c53-434e-9127-4121fff36b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:13:14.038444 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:14.075257 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 641/1294] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:13:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:13:14.342004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba322c5-4603-43ab-b95b-cc10585f2610 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:14.373047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:14.440324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba322c5-4603-43ab-b95b-cc10585f2610 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:14.471793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0d56e-7aa2-4d90-b600-8bc8f9ce27d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:14.472383 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0d56e-7aa2-4d90-b600-8bc8f9ce27d0 None admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:14.472661 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0d56e-7aa2-4d90-b600-8bc8f9ce27d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:14.472928 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0d56e-7aa2-4d90-b600-8bc8f9ce27d0 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:14.473591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0d56e-7aa2-4d90-b600-8bc8f9ce27d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:14.474926 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0d56e-7aa2-4d90-b600-8bc8f9ce27d0 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:14.489538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 654/1295] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Sep 28 12:13:14 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:13:16.630555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3baa6-267a-442d-8a26-99876d29a567 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:16.631090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3baa6-267a-442d-8a26-99876d29a567 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:16.631090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3baa6-267a-442d-8a26-99876d29a567 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:16.632434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 642/1296] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:13:16.638135 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6667680-cb39-437b-a576-cdd37fbc465c None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:16.638563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6667680-cb39-437b-a576-cdd37fbc465c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:16.638683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6667680-cb39-437b-a576-cdd37fbc465c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:16.657062 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-d6667680-cb39-437b-a576-cdd37fbc465c None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:13:16.658256 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d6667680-cb39-437b-a576-cdd37fbc465c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:13:16.732011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 655/1297] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:13:16 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:13:17.005620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3baa6-267a-442d-8a26-99876d29a567 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:17.106935 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3baa6-267a-442d-8a26-99876d29a567 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:17.139730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad9b62-be02-4231-8540-633569d2d9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:17.140360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad9b62-be02-4231-8540-633569d2d9de None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:17.140669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad9b62-be02-4231-8540-633569d2d9de None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:17.140945 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad9b62-be02-4231-8540-633569d2d9de None admin] PATH_INFO: `/v3/services/placement` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:17.141565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad9b62-be02-4231-8540-633569d2d9de None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:17.142926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad9b62-be02-4231-8540-633569d2d9de None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:17.152221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-33ad9b62-be02-4231-8540-633569d2d9de None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 12:13:17.152221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:13:17.152221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:13:17.152221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:13:17.152221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:13:17.152221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:13:17.152221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:13:17.152221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:13:17.152221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 12:13:17.152221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:13:17.152221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:13:17.152221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:13:17.152221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:13:17.152221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:13:17.152221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:13:17.152221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:13:17.152221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:13:17.153393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:13:17.153393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:13:17.153393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:13:17.153393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:13:17.153393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:13:17.153393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:13:17.153393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:13:17.153393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:13:17.153393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:13:17.153393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:13:17.153393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:13:17.153393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:13:17.153393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:13:17.153393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:13:17.153393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:13:17.153393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:13:17.154590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:13:17.154590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:13:17.154590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:13:17.154590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:13:17.154590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:13:17.154590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:13:17.154590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:13:17.154590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:13:17.154590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:13:17.154590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:13:17.154590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:13:17.154590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:13:17.154590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 12:13:17.154590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:13:17.154590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 643/1298] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 12:13:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:13:17.159381 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6667680-cb39-437b-a576-cdd37fbc465c None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:17.177935 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:17.257615 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6667680-cb39-437b-a576-cdd37fbc465c None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:17.290127 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5d634-451e-46a4-9edf-0672bcb79129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:17.290722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5d634-451e-46a4-9edf-0672bcb79129 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:17.291200 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5d634-451e-46a4-9edf-0672bcb79129 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:17.291473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5d634-451e-46a4-9edf-0672bcb79129 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:17.292083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5d634-451e-46a4-9edf-0672bcb79129 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:17.292381 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5d634-451e-46a4-9edf-0672bcb79129 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:17.293757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5d634-451e-46a4-9edf-0672bcb79129 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:17.298643 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 656/1299] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Sep 28 12:13:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:13:17.304547 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad9b62-be02-4231-8540-633569d2d9de None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:17.335924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:17.405011 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad9b62-be02-4231-8540-633569d2d9de None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:17.437523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f25097-3956-457a-a5bb-1fdc3d1bc2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:17.438125 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f25097-3956-457a-a5bb-1fdc3d1bc2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:17.438405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f25097-3956-457a-a5bb-1fdc3d1bc2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:17.438677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f25097-3956-457a-a5bb-1fdc3d1bc2f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:17.439374 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f25097-3956-457a-a5bb-1fdc3d1bc2f1 None admin] RBAC: Adding query filter params (interface=public, service_id=75bddcf34b984f05ad79fbdc81149875, region_id=RegionOne) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:17.439684 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f25097-3956-457a-a5bb-1fdc3d1bc2f1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:17.441044 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f25097-3956-457a-a5bb-1fdc3d1bc2f1 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:17.445908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 644/1300] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Sep 28 12:13:17 2022] GET /identity/v3/endpoints?service_id=75bddcf34b984f05ad79fbdc81149875&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:13:17.452144 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5d634-451e-46a4-9edf-0672bcb79129 None admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:17.531951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:17.549326 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5d634-451e-46a4-9edf-0672bcb79129 None admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:17.581966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c736a9a-67fb-412a-96ed-dbfde4fb68cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:17.582560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c736a9a-67fb-412a-96ed-dbfde4fb68cd None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:17.582840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c736a9a-67fb-412a-96ed-dbfde4fb68cd None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:17.583109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c736a9a-67fb-412a-96ed-dbfde4fb68cd None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:17.583703 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c736a9a-67fb-412a-96ed-dbfde4fb68cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:17.585049 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c736a9a-67fb-412a-96ed-dbfde4fb68cd None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:17.591646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 657/1301] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Sep 28 12:13:17 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:13:19.711383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b931650-1232-4ee8-98e9-363bb5c67f53 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:19.711953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b931650-1232-4ee8-98e9-363bb5c67f53 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:19.711953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b931650-1232-4ee8-98e9-363bb5c67f53 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:19.713426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 645/1302] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:13:19.718549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa751987-20f7-4d24-82a8-3be68f65d92d None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:19.718942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa751987-20f7-4d24-82a8-3be68f65d92d None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:19.719099 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa751987-20f7-4d24-82a8-3be68f65d92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:19.737717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-fa751987-20f7-4d24-82a8-3be68f65d92d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:13:19.738880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-fa751987-20f7-4d24-82a8-3be68f65d92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:13:19.803933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 658/1303] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:13:19 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 12:13:20.084715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b931650-1232-4ee8-98e9-363bb5c67f53 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:20.177892 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:20.187606 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b931650-1232-4ee8-98e9-363bb5c67f53 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:20.220638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceca1a8-eb51-49d1-99b0-275329fdb2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:20.221286 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceca1a8-eb51-49d1-99b0-275329fdb2e1 None admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:20.221566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceca1a8-eb51-49d1-99b0-275329fdb2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:20.221837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceca1a8-eb51-49d1-99b0-275329fdb2e1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:20.222406 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceca1a8-eb51-49d1-99b0-275329fdb2e1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:20.223758 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceca1a8-eb51-49d1-99b0-275329fdb2e1 None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:20.232402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application [None req-6ceca1a8-eb51-49d1-99b0-275329fdb2e1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 12:13:20.232402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:13:20.232402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 12:13:20.232402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 12:13:20.232402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 12:13:20.232402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 12:13:20.232402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 12:13:20.232402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:13:20.232402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 12:13:20.232402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:13:20.232402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 12:13:20.232402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:13:20.232402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 12:13:20.232402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 12:13:20.232402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 12:13:20.232402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 12:13:20.232402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 12:13:20.233582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 12:13:20.233582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 12:13:20.233582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 12:13:20.233582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 12:13:20.233582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 12:13:20.233582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 12:13:20.233582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 12:13:20.233582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 12:13:20.233582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 12:13:20.233582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 12:13:20.233582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 12:13:20.233582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 12:13:20.233582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 12:13:20.233582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 12:13:20.233582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 12:13:20.233582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 12:13:20.234777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 12:13:20.234777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application with Lock( Sep 28 12:13:20.234777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 12:13:20.234777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self._enter() Sep 28 12:13:20.234777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 12:13:20.234777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 12:13:20.234777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 12:13:20.234777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application return self.creator() Sep 28 12:13:20.234777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 12:13:20.234777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application created_value = creator( Sep 28 12:13:20.234777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 12:13:20.234777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 12:13:20.234777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 12:13:20.234777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: ERROR keystone.server.flask.application Sep 28 12:13:20.234777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 646/1304] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Sep 28 12:13:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 12:13:20.239709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa751987-20f7-4d24-82a8-3be68f65d92d None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:20.312461 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:20.334808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa751987-20f7-4d24-82a8-3be68f65d92d None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:20.367904 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f368c-3804-4b4c-80f8-7eed58333143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:20.368505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f368c-3804-4b4c-80f8-7eed58333143 None admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:20.368797 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f368c-3804-4b4c-80f8-7eed58333143 None admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:20.369179 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f368c-3804-4b4c-80f8-7eed58333143 None admin] PATH_INFO: `/v3/services` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:20.369794 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f368c-3804-4b4c-80f8-7eed58333143 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:13:20.370092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f368c-3804-4b4c-80f8-7eed58333143 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:20.371446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f368c-3804-4b4c-80f8-7eed58333143 None admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:20.376345 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 659/1305] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Sep 28 12:13:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:13:20.382176 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceca1a8-eb51-49d1-99b0-275329fdb2e1 None admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:13:20.479263 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceca1a8-eb51-49d1-99b0-275329fdb2e1 None admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:13:20.512351 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f927cde-dd27-4a2f-b390-e1e62d76fb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:13:20.512936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f927cde-dd27-4a2f-b390-e1e62d76fb1d None admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:13:20.513240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f927cde-dd27-4a2f-b390-e1e62d76fb1d None admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:13:20.513511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f927cde-dd27-4a2f-b390-e1e62d76fb1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:13:20.514169 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f927cde-dd27-4a2f-b390-e1e62d76fb1d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:13:20.515508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f927cde-dd27-4a2f-b390-e1e62d76fb1d None admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:13:20.534521 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 647/1306] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:13:20 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:13:30.373656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=30 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:13:30.530549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:24.540022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923f970-569b-4834-a326-191bfc34218c None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:24.540495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923f970-569b-4834-a326-191bfc34218c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:24.540495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923f970-569b-4834-a326-191bfc34218c None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:24.547900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d790d6-f194-44e2-8835-b24ae4bb0bd4 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:24.549114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d790d6-f194-44e2-8835-b24ae4bb0bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:24.549820 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d790d6-f194-44e2-8835-b24ae4bb0bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:24.555922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 660/1307] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:14:24 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:14:24.660223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-06d790d6-f194-44e2-8835-b24ae4bb0bd4 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:14:24.662272 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-06d790d6-f194-44e2-8835-b24ae4bb0bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:24.800239 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 648/1308] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:14:25.097480 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c47c4-ffd6-436d-8641-340f6c5435f3 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:25.097856 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c47c4-ffd6-436d-8641-340f6c5435f3 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:25.098199 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c47c4-ffd6-436d-8641-340f6c5435f3 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:25.099642 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 661/1309] 10.222.0.60 () {54 vars in 964 bytes} [Wed Sep 28 12:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:14:25.106119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790cbb6-2fc6-446a-b2fe-2be061f873c0 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:25.106408 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790cbb6-2fc6-446a-b2fe-2be061f873c0 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:25.106686 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790cbb6-2fc6-446a-b2fe-2be061f873c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:25.200134 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-7790cbb6-2fc6-446a-b2fe-2be061f873c0 None None] MFA Rules not processed for user `ba75586d3a7d4278ad6c0c31a1bed3df`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:14:25.201354 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7790cbb6-2fc6-446a-b2fe-2be061f873c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:25.223494 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:25.284388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 649/1310] 10.222.0.60 () {58 vars in 1061 bytes} [Wed Sep 28 12:14:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:14:25.294063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c47c4-ffd6-436d-8641-340f6c5435f3 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:14:25.295792 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a45c47c4-ffd6-436d-8641-340f6c5435f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:25.417450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c47c4-ffd6-436d-8641-340f6c5435f3 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:14:25.498466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c40567-700b-4cf7-803b-1970d227d5b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:14:25.499069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c40567-700b-4cf7-803b-1970d227d5b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:25.499339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c40567-700b-4cf7-803b-1970d227d5b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:25.499604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c40567-700b-4cf7-803b-1970d227d5b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:25.501224 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-12c40567-700b-4cf7-803b-1970d227d5b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:25.562990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:25.624750 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c40567-700b-4cf7-803b-1970d227d5b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:14:25.629351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c40567-700b-4cf7-803b-1970d227d5b6 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:14:25.654917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 662/1311] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:14:28.102616 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80563768-6ed2-4bcb-b8d7-62ae2a166e11 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:28.103146 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80563768-6ed2-4bcb-b8d7-62ae2a166e11 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:28.103146 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80563768-6ed2-4bcb-b8d7-62ae2a166e11 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:28.104562 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 650/1312] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:14:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:14:28.110090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfe69d-4498-4761-b8a1-6a8ed1c0a770 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:28.110523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfe69d-4498-4761-b8a1-6a8ed1c0a770 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:28.110722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfe69d-4498-4761-b8a1-6a8ed1c0a770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:28.194151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-f7bfe69d-4498-4761-b8a1-6a8ed1c0a770 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:14:28.195591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-f7bfe69d-4498-4761-b8a1-6a8ed1c0a770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:28.290890 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 663/1313] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:14:28.572037 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80563768-6ed2-4bcb-b8d7-62ae2a166e11 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:14:28.573590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-80563768-6ed2-4bcb-b8d7-62ae2a166e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:28.687440 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80563768-6ed2-4bcb-b8d7-62ae2a166e11 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:14:28.762805 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:28.765518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c14dc-522d-4d60-897e-8d1d2c0563c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:14:28.766548 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c14dc-522d-4d60-897e-8d1d2c0563c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:28.766999 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c14dc-522d-4d60-897e-8d1d2c0563c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:28.767324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c14dc-522d-4d60-897e-8d1d2c0563c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:28.769291 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-557c14dc-522d-4d60-897e-8d1d2c0563c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:28.878484 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c14dc-522d-4d60-897e-8d1d2c0563c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:14:28.879825 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c14dc-522d-4d60-897e-8d1d2c0563c3 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:14:28.897358 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 651/1314] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:14:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:14:31.369111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4b330-5dea-47e8-a9a3-1958bedfa98a None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:31.369696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4b330-5dea-47e8-a9a3-1958bedfa98a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:31.369696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4b330-5dea-47e8-a9a3-1958bedfa98a None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:31.371008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 664/1315] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:14:31.376200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2efab-774c-4444-8553-0880f268e859 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:31.376598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2efab-774c-4444-8553-0880f268e859 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:31.376748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2efab-774c-4444-8553-0880f268e859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:31.443819 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-24e2efab-774c-4444-8553-0880f268e859 None None] MFA Rules not processed for user `a45fb43467b9498eb1b647735d0b88b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:14:31.445244 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-24e2efab-774c-4444-8553-0880f268e859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:31.497107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:31.541433 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 652/1316] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:14:31 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:14:31.841368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c308d87-f093-473e-acf8-2c44065a3476 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:31.841650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c308d87-f093-473e-acf8-2c44065a3476 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:31.841923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c308d87-f093-473e-acf8-2c44065a3476 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:31.843253 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 665/1317] 10.222.0.60 () {54 vars in 964 bytes} [Wed Sep 28 12:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:14:31.848882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056c021-be3a-4df1-b078-47a981d39244 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:31.849296 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056c021-be3a-4df1-b078-47a981d39244 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:31.849567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056c021-be3a-4df1-b078-47a981d39244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:31.925292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-6056c021-be3a-4df1-b078-47a981d39244 None None] MFA Rules not processed for user `ba75586d3a7d4278ad6c0c31a1bed3df`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:14:31.926495 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-6056c021-be3a-4df1-b078-47a981d39244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:32.002665 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 653/1318] 10.222.0.60 () {58 vars in 1061 bytes} [Wed Sep 28 12:14:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:14:32.012303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c308d87-f093-473e-acf8-2c44065a3476 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:14:32.013801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9c308d87-f093-473e-acf8-2c44065a3476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:32.023851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:32.115723 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c308d87-f093-473e-acf8-2c44065a3476 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:14:32.190781 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7e3fe-0775-4075-b163-3231683e40a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:14:32.191381 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7e3fe-0775-4075-b163-3231683e40a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:32.191657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7e3fe-0775-4075-b163-3231683e40a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:32.191924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7e3fe-0775-4075-b163-3231683e40a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:32.193523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a9d7e3fe-0775-4075-b163-3231683e40a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:32.254913 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:32.300635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7e3fe-0775-4075-b163-3231683e40a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:14:32.302003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7e3fe-0775-4075-b163-3231683e40a9 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:14:32.320268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 666/1319] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:14:32 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:14:35.446578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3696be-cabf-4734-be87-9dd297b4baad None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:35.447146 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3696be-cabf-4734-be87-9dd297b4baad None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:35.447146 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3696be-cabf-4734-be87-9dd297b4baad None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:35.448484 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 654/1320] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:14:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:14:35.453720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ab23f-ecc3-4b79-b7ad-9e76d9ffd67a None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:35.454155 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ab23f-ecc3-4b79-b7ad-9e76d9ffd67a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:35.454262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ab23f-ecc3-4b79-b7ad-9e76d9ffd67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:35.525343 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-3f5ab23f-ecc3-4b79-b7ad-9e76d9ffd67a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:14:35.526540 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-3f5ab23f-ecc3-4b79-b7ad-9e76d9ffd67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:35.621397 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 667/1321] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:14:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:14:35.911173 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3696be-cabf-4734-be87-9dd297b4baad None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:14:35.912727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0c3696be-cabf-4734-be87-9dd297b4baad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:35.963385 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:36.016225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3696be-cabf-4734-be87-9dd297b4baad None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:14:36.087563 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b2a93-a3ee-434f-a8a1-f9f10b337b45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:14:36.088165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b2a93-a3ee-434f-a8a1-f9f10b337b45 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:36.088446 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b2a93-a3ee-434f-a8a1-f9f10b337b45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:36.088720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b2a93-a3ee-434f-a8a1-f9f10b337b45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:36.090409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0e7b2a93-a3ee-434f-a8a1-f9f10b337b45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:36.185580 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:36.199021 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b2a93-a3ee-434f-a8a1-f9f10b337b45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:14:36.200481 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b2a93-a3ee-434f-a8a1-f9f10b337b45 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:14:36.218491 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 655/1322] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:14:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:14:39.027129 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889f711-1748-4e04-9b12-f085c1ced94e None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:39.027700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889f711-1748-4e04-9b12-f085c1ced94e None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:39.027700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889f711-1748-4e04-9b12-f085c1ced94e None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:39.029007 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 668/1323] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:14:39.034505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c400ce-372a-4014-8249-88dfe94889bd None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:39.034898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c400ce-372a-4014-8249-88dfe94889bd None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:39.035050 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c400ce-372a-4014-8249-88dfe94889bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:39.111586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-50c400ce-372a-4014-8249-88dfe94889bd None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:14:39.112829 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-50c400ce-372a-4014-8249-88dfe94889bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:39.228112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 656/1324] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:14:39.512044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889f711-1748-4e04-9b12-f085c1ced94e None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:14:39.513665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-4889f711-1748-4e04-9b12-f085c1ced94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:39.533236 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:39.618029 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889f711-1748-4e04-9b12-f085c1ced94e None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:14:39.693440 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdbded-ddb5-478c-bb72-57a44ea5745a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:14:39.694049 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdbded-ddb5-478c-bb72-57a44ea5745a service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:39.694356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdbded-ddb5-478c-bb72-57a44ea5745a service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:39.694628 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdbded-ddb5-478c-bb72-57a44ea5745a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:39.696207 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-90fdbded-ddb5-478c-bb72-57a44ea5745a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:39.769563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:39.808027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdbded-ddb5-478c-bb72-57a44ea5745a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:14:39.809536 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdbded-ddb5-478c-bb72-57a44ea5745a service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:14:39.828146 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 669/1325] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:14:42.055621 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cdefa-5b3b-4045-881c-392bf4e41b96 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:42.056156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cdefa-5b3b-4045-881c-392bf4e41b96 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:42.056156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cdefa-5b3b-4045-881c-392bf4e41b96 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:42.057499 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 657/1326] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:14:42.062955 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26712716-02a7-49ef-b1ef-e803524667af None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:42.063360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26712716-02a7-49ef-b1ef-e803524667af None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:42.063538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26712716-02a7-49ef-b1ef-e803524667af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:42.130854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-26712716-02a7-49ef-b1ef-e803524667af None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:14:42.132624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-26712716-02a7-49ef-b1ef-e803524667af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:42.238275 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 670/1327] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:14:42.518661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840cdefa-5b3b-4045-881c-392bf4e41b96 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:14:42.520170 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-840cdefa-5b3b-4045-881c-392bf4e41b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:42.580347 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:42.625170 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840cdefa-5b3b-4045-881c-392bf4e41b96 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:14:42.700291 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf28056-bfaf-445e-a9f7-aa8edef0b2ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:14:42.700898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf28056-bfaf-445e-a9f7-aa8edef0b2ed service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:42.701203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf28056-bfaf-445e-a9f7-aa8edef0b2ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:42.701481 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf28056-bfaf-445e-a9f7-aa8edef0b2ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:42.703059 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-cbf28056-bfaf-445e-a9f7-aa8edef0b2ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:42.810745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:42.813336 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf28056-bfaf-445e-a9f7-aa8edef0b2ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:14:42.815357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf28056-bfaf-445e-a9f7-aa8edef0b2ed service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:14:42.838069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 658/1328] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:14:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:14:45.074342 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15239170-c2df-4e09-b8a7-6e243e1a9b56 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:45.074846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15239170-c2df-4e09-b8a7-6e243e1a9b56 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:45.074846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15239170-c2df-4e09-b8a7-6e243e1a9b56 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:45.076124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 671/1329] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:14:45.081650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02156b6-5a51-4783-af11-eae28b0f6a2c None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:45.082079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02156b6-5a51-4783-af11-eae28b0f6a2c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:45.082195 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02156b6-5a51-4783-af11-eae28b0f6a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:45.150140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-a02156b6-5a51-4783-af11-eae28b0f6a2c None None] MFA Rules not processed for user `a45fb43467b9498eb1b647735d0b88b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:14:45.151351 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-a02156b6-5a51-4783-af11-eae28b0f6a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:45.249303 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 659/1330] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:14:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:14:45.539022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15239170-c2df-4e09-b8a7-6e243e1a9b56 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:14:45.540544 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-15239170-c2df-4e09-b8a7-6e243e1a9b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:45.573353 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:45.648894 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15239170-c2df-4e09-b8a7-6e243e1a9b56 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:14:45.721864 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cee9b5-e1ca-4b9b-b93c-900c87824f4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:14:45.722464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cee9b5-e1ca-4b9b-b93c-900c87824f4d service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:45.722739 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cee9b5-e1ca-4b9b-b93c-900c87824f4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:45.723008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cee9b5-e1ca-4b9b-b93c-900c87824f4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:45.724570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-04cee9b5-e1ca-4b9b-b93c-900c87824f4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:45.805368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:45.848360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cee9b5-e1ca-4b9b-b93c-900c87824f4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:14:45.849842 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cee9b5-e1ca-4b9b-b93c-900c87824f4d service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:14:45.870047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 672/1331] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:14:45 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:14:48.351925 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1a1f6-3fe4-452a-bb12-9d344010a0a9 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:48.352491 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1a1f6-3fe4-452a-bb12-9d344010a0a9 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:48.352491 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1a1f6-3fe4-452a-bb12-9d344010a0a9 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:48.353860 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 660/1332] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:14:48.359124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5c25f-00bb-407d-85d6-31355d203337 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:48.359537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5c25f-00bb-407d-85d6-31355d203337 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:48.359926 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5c25f-00bb-407d-85d6-31355d203337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:48.439058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-64d5c25f-00bb-407d-85d6-31355d203337 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:14:48.440914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-64d5c25f-00bb-407d-85d6-31355d203337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:48.554533 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 673/1333] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:14:48.846152 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1a1f6-3fe4-452a-bb12-9d344010a0a9 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:14:48.847685 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-47c1a1f6-3fe4-452a-bb12-9d344010a0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:48.916469 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:48.950597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1a1f6-3fe4-452a-bb12-9d344010a0a9 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:14:49.023356 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6a59e-c098-46ec-8dff-a92da42b6870 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:14:49.023948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6a59e-c098-46ec-8dff-a92da42b6870 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:49.024223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6a59e-c098-46ec-8dff-a92da42b6870 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:49.024492 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6a59e-c098-46ec-8dff-a92da42b6870 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:49.026087 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-aee6a59e-c098-46ec-8dff-a92da42b6870 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:49.133776 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6a59e-c098-46ec-8dff-a92da42b6870 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:14:49.135133 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6a59e-c098-46ec-8dff-a92da42b6870 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:14:49.145416 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:49.154093 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 661/1334] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:14:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:14:51.364389 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e1c0-895f-4ff2-83f5-61e977d6b3ac None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:51.364933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e1c0-895f-4ff2-83f5-61e977d6b3ac None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:51.364933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5e1c0-895f-4ff2-83f5-61e977d6b3ac None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:51.366390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 674/1335] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:14:51.371731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0135b5d-5acb-4170-83ad-817559ccaeee None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:51.372128 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0135b5d-5acb-4170-83ad-817559ccaeee None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:51.372293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0135b5d-5acb-4170-83ad-817559ccaeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:51.437863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-b0135b5d-5acb-4170-83ad-817559ccaeee None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:14:51.439051 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b0135b5d-5acb-4170-83ad-817559ccaeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:51.538659 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 662/1336] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:14:51.824114 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5e1c0-895f-4ff2-83f5-61e977d6b3ac None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:14:51.825678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d6b5e1c0-895f-4ff2-83f5-61e977d6b3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:51.865493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:51.927851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5e1c0-895f-4ff2-83f5-61e977d6b3ac None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:14:51.999249 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388da30-13f3-4960-8090-c1457932a4db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:14:51.999843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388da30-13f3-4960-8090-c1457932a4db service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:52.000117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388da30-13f3-4960-8090-c1457932a4db service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:52.000381 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388da30-13f3-4960-8090-c1457932a4db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:52.001949 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e388da30-13f3-4960-8090-c1457932a4db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:52.083901 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:52.108705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388da30-13f3-4960-8090-c1457932a4db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:14:52.110400 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388da30-13f3-4960-8090-c1457932a4db service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:14:52.128610 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 675/1337] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:14:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:14:54.581527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d2632-c892-4601-a129-34b266d96e96 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:54.582072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d2632-c892-4601-a129-34b266d96e96 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:54.582072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d2632-c892-4601-a129-34b266d96e96 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:54.583411 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 663/1338] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:14:54.589417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b11824-5a3a-469d-b1f3-9f9930e8a4b0 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:54.589825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b11824-5a3a-469d-b1f3-9f9930e8a4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:54.589973 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b11824-5a3a-469d-b1f3-9f9930e8a4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:54.655406 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-99b11824-5a3a-469d-b1f3-9f9930e8a4b0 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:14:54.656628 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-99b11824-5a3a-469d-b1f3-9f9930e8a4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:54.754024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 676/1339] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:14:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:14:55.035189 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d2632-c892-4601-a129-34b266d96e96 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:14:55.036737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-407d2632-c892-4601-a129-34b266d96e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:55.111760 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:55.140049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d2632-c892-4601-a129-34b266d96e96 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:14:55.212793 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaed9b8-ef0e-46e3-87e3-908415e15615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:14:55.213438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaed9b8-ef0e-46e3-87e3-908415e15615 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:55.213720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaed9b8-ef0e-46e3-87e3-908415e15615 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:55.213991 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaed9b8-ef0e-46e3-87e3-908415e15615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:55.215604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-ccaed9b8-ef0e-46e3-87e3-908415e15615 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:55.324791 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaed9b8-ef0e-46e3-87e3-908415e15615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:14:55.326190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaed9b8-ef0e-46e3-87e3-908415e15615 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:14:55.344488 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 664/1340] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:14:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:14:59.405826 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f6d0f-e8dd-4440-98f3-077f07732d25 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:59.406376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f6d0f-e8dd-4440-98f3-077f07732d25 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:59.406376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f6d0f-e8dd-4440-98f3-077f07732d25 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:59.407930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 677/1341] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:14:59.413751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97605d2d-823c-49f0-9744-ed0805bc8c1b None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:14:59.414240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97605d2d-823c-49f0-9744-ed0805bc8c1b None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:14:59.414321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97605d2d-823c-49f0-9744-ed0805bc8c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:14:59.420390 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:59.486799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-97605d2d-823c-49f0-9744-ed0805bc8c1b None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:14:59.487938 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-97605d2d-823c-49f0-9744-ed0805bc8c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:59.593432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 665/1342] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:14:59.882362 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f6d0f-e8dd-4440-98f3-077f07732d25 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:14:59.883953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-bd5f6d0f-e8dd-4440-98f3-077f07732d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:14:59.932945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:14:59.987830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f6d0f-e8dd-4440-98f3-077f07732d25 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:15:00.061210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66791b63-397f-481f-9e80-95b90cbc892e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:15:00.061740 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66791b63-397f-481f-9e80-95b90cbc892e service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:00.062016 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66791b63-397f-481f-9e80-95b90cbc892e service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:00.062284 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66791b63-397f-481f-9e80-95b90cbc892e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:00.063832 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-66791b63-397f-481f-9e80-95b90cbc892e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:00.158431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:00.175605 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66791b63-397f-481f-9e80-95b90cbc892e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:15:00.177030 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66791b63-397f-481f-9e80-95b90cbc892e service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:15:00.195879 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 678/1343] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:14:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:15:02.995220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6631b-27e9-45af-8342-c94a1d4ba12d None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:02.995765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6631b-27e9-45af-8342-c94a1d4ba12d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:02.995765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6631b-27e9-45af-8342-c94a1d4ba12d None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:02.997100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 666/1344] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:03.002348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533844c-2300-400d-b542-f987ea368ed8 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:03.002817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533844c-2300-400d-b542-f987ea368ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:03.003047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533844c-2300-400d-b542-f987ea368ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:03.068212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7533844c-2300-400d-b542-f987ea368ed8 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:03.069477 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7533844c-2300-400d-b542-f987ea368ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:03.164325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 679/1345] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:15:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:03.447184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6631b-27e9-45af-8342-c94a1d4ba12d None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:15:03.448738 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7af6631b-27e9-45af-8342-c94a1d4ba12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:03.540585 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:03.560318 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6631b-27e9-45af-8342-c94a1d4ba12d None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:15:03.637465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8760ff7-4c59-4258-bc9a-7c62a48cb83f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:15:03.638074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8760ff7-4c59-4258-bc9a-7c62a48cb83f service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:03.638335 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8760ff7-4c59-4258-bc9a-7c62a48cb83f service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:03.638601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8760ff7-4c59-4258-bc9a-7c62a48cb83f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:03.640170 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d8760ff7-4c59-4258-bc9a-7c62a48cb83f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:03.750602 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8760ff7-4c59-4258-bc9a-7c62a48cb83f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:15:03.751945 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8760ff7-4c59-4258-bc9a-7c62a48cb83f service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:15:03.771569 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 667/1346] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:15:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:15:07.899946 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b63d1-5d9a-4e6e-a04a-c1aebe2dba95 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:07.900469 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b63d1-5d9a-4e6e-a04a-c1aebe2dba95 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:07.900581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b63d1-5d9a-4e6e-a04a-c1aebe2dba95 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:07.901940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 680/1347] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:07.907294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb503fc-1666-4b34-bace-dc5cfa5cdca1 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:07.908068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb503fc-1666-4b34-bace-dc5cfa5cdca1 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:07.908344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb503fc-1666-4b34-bace-dc5cfa5cdca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:07.925215 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:07.980736 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-efb503fc-1666-4b34-bace-dc5cfa5cdca1 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:07.982112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-efb503fc-1666-4b34-bace-dc5cfa5cdca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:08.094309 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 668/1348] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:08.377970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b63d1-5d9a-4e6e-a04a-c1aebe2dba95 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:15:08.379530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-c37b63d1-5d9a-4e6e-a04a-c1aebe2dba95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:08.436863 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:08.481789 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b63d1-5d9a-4e6e-a04a-c1aebe2dba95 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:15:08.553994 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fccc8-3687-4d76-9445-4a600ec22e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:15:08.554594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fccc8-3687-4d76-9445-4a600ec22e84 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:08.554870 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fccc8-3687-4d76-9445-4a600ec22e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:08.555136 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fccc8-3687-4d76-9445-4a600ec22e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:08.556702 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-754fccc8-3687-4d76-9445-4a600ec22e84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:08.659713 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:08.662987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fccc8-3687-4d76-9445-4a600ec22e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:15:08.664712 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fccc8-3687-4d76-9445-4a600ec22e84 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:15:08.683003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 681/1349] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:15:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:15:13.592540 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9d62c-c92b-4ac7-a090-056f6b2cb370 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:13.593009 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9d62c-c92b-4ac7-a090-056f6b2cb370 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:13.593088 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9d62c-c92b-4ac7-a090-056f6b2cb370 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:13.594627 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 669/1350] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 12:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:13.600590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1f9b3-c347-467a-ad77-bd38f1270def None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:13.601066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1f9b3-c347-467a-ad77-bd38f1270def None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:13.601368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e1f9b3-c347-467a-ad77-bd38f1270def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:13.755426 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-97e1f9b3-c347-467a-ad77-bd38f1270def None None] MFA Rules not processed for user `9a2322b1a76d40e49545f5677e247489`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:13.757415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-97e1f9b3-c347-467a-ad77-bd38f1270def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:13.883005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 682/1351] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 12:15:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:17.970665 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc6566-e670-4a40-999b-ecda5ec933c6 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:17.971178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc6566-e670-4a40-999b-ecda5ec933c6 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:17.971178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc6566-e670-4a40-999b-ecda5ec933c6 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:17.972687 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 670/1352] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 12:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:17.978477 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe72f6-9fc8-4dc1-b0a2-4da40b1b02cc None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:17.979464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe72f6-9fc8-4dc1-b0a2-4da40b1b02cc None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:17.979983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe72f6-9fc8-4dc1-b0a2-4da40b1b02cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:18.044261 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:18.072410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-50fe72f6-9fc8-4dc1-b0a2-4da40b1b02cc None None] MFA Rules not processed for user `9a2322b1a76d40e49545f5677e247489`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:18.073803 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-50fe72f6-9fc8-4dc1-b0a2-4da40b1b02cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:18.092486 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=59 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:18.200041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 683/1353] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 12:15:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:19.605071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146eeee4-5198-4e02-926b-f6b19ab8573f None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:19.605628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146eeee4-5198-4e02-926b-f6b19ab8573f None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:19.605628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146eeee4-5198-4e02-926b-f6b19ab8573f None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:19.606975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 671/1354] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 12:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:19.612384 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77212548-1e6d-44d2-ba7a-9984780676b0 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:19.612793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77212548-1e6d-44d2-ba7a-9984780676b0 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:19.613001 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77212548-1e6d-44d2-ba7a-9984780676b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:19.699648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-77212548-1e6d-44d2-ba7a-9984780676b0 None None] MFA Rules not processed for user `9a2322b1a76d40e49545f5677e247489`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:19.701090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-77212548-1e6d-44d2-ba7a-9984780676b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:19.749212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:19.819252 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 684/1355] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 12:15:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:23.410553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b387a9f-9f57-448e-961b-033f288dc61a None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:23.411049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b387a9f-9f57-448e-961b-033f288dc61a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:23.411049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b387a9f-9f57-448e-961b-033f288dc61a None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:23.412932 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 672/1356] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:23.418394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b52f54-6dd4-466c-a057-98a7e1eb8552 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:23.418858 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b52f54-6dd4-466c-a057-98a7e1eb8552 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:23.418955 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b52f54-6dd4-466c-a057-98a7e1eb8552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:23.498043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-a0b52f54-6dd4-466c-a057-98a7e1eb8552 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:23.499228 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a0b52f54-6dd4-466c-a057-98a7e1eb8552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:23.608285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 685/1357] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:15:23.921894 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb98e5e-b369-4895-b5d2-757824761ff4 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:23.922425 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb98e5e-b369-4895-b5d2-757824761ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:23.922898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb98e5e-b369-4895-b5d2-757824761ff4 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:23.924613 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 673/1358] 10.222.0.60 () {54 vars in 961 bytes} [Wed Sep 28 12:15:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:23.934150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f07a6-2cd3-4af5-83f1-1902a0cd595e None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:23.934720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f07a6-2cd3-4af5-83f1-1902a0cd595e None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:23.935263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f07a6-2cd3-4af5-83f1-1902a0cd595e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:23.956198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:24.024618 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-177f07a6-2cd3-4af5-83f1-1902a0cd595e None None] MFA Rules not processed for user `982b82e8ac424318a934f9149fb91706`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:24.025847 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-177f07a6-2cd3-4af5-83f1-1902a0cd595e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:24.149662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 686/1359] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Sep 28 12:15:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:24.157483 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb98e5e-b369-4895-b5d2-757824761ff4 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:15:24.159232 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-bdb98e5e-b369-4895-b5d2-757824761ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:24.328903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb98e5e-b369-4895-b5d2-757824761ff4 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:15:24.410574 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:24.456747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2adfdf-2a95-406e-b0b4-113885fe6d3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:15:24.457393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2adfdf-2a95-406e-b0b4-113885fe6d3f service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:24.457676 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2adfdf-2a95-406e-b0b4-113885fe6d3f service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:24.457948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2adfdf-2a95-406e-b0b4-113885fe6d3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:24.460630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2d2adfdf-2a95-406e-b0b4-113885fe6d3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:24.570263 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2adfdf-2a95-406e-b0b4-113885fe6d3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:15:24.571955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2adfdf-2a95-406e-b0b4-113885fe6d3f service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:15:24.592641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 674/1360] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:15:27.002699 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34402b4e-0b3a-4a3a-b7eb-a7b9e075dae6 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:27.003248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34402b4e-0b3a-4a3a-b7eb-a7b9e075dae6 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:27.003248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34402b4e-0b3a-4a3a-b7eb-a7b9e075dae6 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:27.004780 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 687/1361] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:27.010714 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08560d6-826c-4d5c-99a4-e7d7b2f421d0 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:27.011184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08560d6-826c-4d5c-99a4-e7d7b2f421d0 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:27.011391 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08560d6-826c-4d5c-99a4-e7d7b2f421d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:27.093605 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-a08560d6-826c-4d5c-99a4-e7d7b2f421d0 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:27.094909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-a08560d6-826c-4d5c-99a4-e7d7b2f421d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:27.138620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:27.215453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 675/1362] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:27.517803 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34402b4e-0b3a-4a3a-b7eb-a7b9e075dae6 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:15:27.519609 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-34402b4e-0b3a-4a3a-b7eb-a7b9e075dae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:27.608402 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:27.718430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34402b4e-0b3a-4a3a-b7eb-a7b9e075dae6 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:15:27.837046 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb71c3-de50-49b1-b815-dfd8cacc4d2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:15:27.837683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb71c3-de50-49b1-b815-dfd8cacc4d2b service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:27.837959 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb71c3-de50-49b1-b815-dfd8cacc4d2b service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:27.838230 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb71c3-de50-49b1-b815-dfd8cacc4d2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:27.839833 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-8dbb71c3-de50-49b1-b815-dfd8cacc4d2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:27.864739 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:27.941073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb71c3-de50-49b1-b815-dfd8cacc4d2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:15:27.943513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb71c3-de50-49b1-b815-dfd8cacc4d2b service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:15:27.963517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 688/1363] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:15:30.488462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc5992-6a5a-429f-9c99-0784a2a6f2c1 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:30.490214 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc5992-6a5a-429f-9c99-0784a2a6f2c1 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:30.490688 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc5992-6a5a-429f-9c99-0784a2a6f2c1 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:30.492164 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 676/1364] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:15:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:30.497948 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215118a-37de-4f90-8144-b90d2942f434 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:30.498417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215118a-37de-4f90-8144-b90d2942f434 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:30.498609 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215118a-37de-4f90-8144-b90d2942f434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:30.571676 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-d215118a-37de-4f90-8144-b90d2942f434 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:30.572891 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d215118a-37de-4f90-8144-b90d2942f434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:30.675914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:30.685475 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 689/1365] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:30.978552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5811f-8edc-497c-a538-976e2198d7dc None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:30.978958 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5811f-8edc-497c-a538-976e2198d7dc None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:30.979334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5811f-8edc-497c-a538-976e2198d7dc None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:30.980878 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 677/1366] 10.222.0.60 () {54 vars in 961 bytes} [Wed Sep 28 12:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:30.986388 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fcd17-2670-4bd5-a347-f9c434960533 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:30.986862 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fcd17-2670-4bd5-a347-f9c434960533 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:30.987137 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fcd17-2670-4bd5-a347-f9c434960533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:31.067923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-ef6fcd17-2670-4bd5-a347-f9c434960533 None None] MFA Rules not processed for user `982b82e8ac424318a934f9149fb91706`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:31.069148 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-ef6fcd17-2670-4bd5-a347-f9c434960533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:31.210395 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 690/1367] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Sep 28 12:15:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:31.213776 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5811f-8edc-497c-a538-976e2198d7dc None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:15:31.215414 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-17b5811f-8edc-497c-a538-976e2198d7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:31.379212 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5811f-8edc-497c-a538-976e2198d7dc None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:15:31.401002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:31.501508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14ad4e-dc9a-4990-ba50-666a993936b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:15:31.502183 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14ad4e-dc9a-4990-ba50-666a993936b7 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:31.502444 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14ad4e-dc9a-4990-ba50-666a993936b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:31.502615 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14ad4e-dc9a-4990-ba50-666a993936b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:31.504149 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-af14ad4e-dc9a-4990-ba50-666a993936b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:31.607909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14ad4e-dc9a-4990-ba50-666a993936b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:15:31.609792 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14ad4e-dc9a-4990-ba50-666a993936b7 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:15:31.631430 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 678/1368] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:15:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:15:31.682748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f19b5-a793-4e25-ad9d-8823fcab0a59 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:31.683255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f19b5-a793-4e25-ad9d-8823fcab0a59 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:31.683355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f19b5-a793-4e25-ad9d-8823fcab0a59 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:31.684793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 691/1369] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:31.690405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21259b9d-268d-4b3d-918d-77fa99f2935e None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:31.690720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21259b9d-268d-4b3d-918d-77fa99f2935e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:31.691072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21259b9d-268d-4b3d-918d-77fa99f2935e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:31.728423 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:31.777674 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-21259b9d-268d-4b3d-918d-77fa99f2935e None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:31.779079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-21259b9d-268d-4b3d-918d-77fa99f2935e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:31.901422 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 679/1370] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:32.245785 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b875d-dad3-4e97-95f2-8dd80f377dc8 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:32.245785 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b875d-dad3-4e97-95f2-8dd80f377dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:32.245785 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b875d-dad3-4e97-95f2-8dd80f377dc8 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:32.247847 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 692/1371] 10.222.0.60 () {54 vars in 963 bytes} [Wed Sep 28 12:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:32.251675 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93393320-9427-4056-bf24-7ab1ce69d41c None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:32.251982 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93393320-9427-4056-bf24-7ab1ce69d41c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:32.252279 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93393320-9427-4056-bf24-7ab1ce69d41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:32.339589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-93393320-9427-4056-bf24-7ab1ce69d41c None None] MFA Rules not processed for user `2354a4fd72cc42c58d6830eecb1e6253`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:32.340889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-93393320-9427-4056-bf24-7ab1ce69d41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:32.391482 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:32.425360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 680/1372] 10.222.0.60 () {58 vars in 1060 bytes} [Wed Sep 28 12:15:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:32.432572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b875d-dad3-4e97-95f2-8dd80f377dc8 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:15:32.434129 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-f01b875d-dad3-4e97-95f2-8dd80f377dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:32.490251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:32.550978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01b875d-dad3-4e97-95f2-8dd80f377dc8 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:15:32.648674 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f9bdc1-4243-428f-b1a9-ea1ac591f675 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:15:32.650134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9bdc1-4243-428f-b1a9-ea1ac591f675 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:32.650709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9bdc1-4243-428f-b1a9-ea1ac591f675 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:32.650980 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f9bdc1-4243-428f-b1a9-ea1ac591f675 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:32.653264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a5f9bdc1-4243-428f-b1a9-ea1ac591f675 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:32.785651 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:32.803168 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f9bdc1-4243-428f-b1a9-ea1ac591f675 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:15:32.805276 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f9bdc1-4243-428f-b1a9-ea1ac591f675 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:15:32.831688 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 693/1373] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:15:33.923068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04f641-67df-4de5-90b8-38298f52d4c2 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:33.923663 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04f641-67df-4de5-90b8-38298f52d4c2 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:33.923663 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04f641-67df-4de5-90b8-38298f52d4c2 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:33.924950 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 681/1374] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:33.930436 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bac112-6238-4c68-a462-74eff34a6f0a None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:33.930900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bac112-6238-4c68-a462-74eff34a6f0a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:33.931113 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bac112-6238-4c68-a462-74eff34a6f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:34.002206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-77bac112-6238-4c68-a462-74eff34a6f0a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:34.003415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-77bac112-6238-4c68-a462-74eff34a6f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:34.098664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 694/1375] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:34.389509 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04f641-67df-4de5-90b8-38298f52d4c2 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:15:34.391201 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0c04f641-67df-4de5-90b8-38298f52d4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:34.551181 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04f641-67df-4de5-90b8-38298f52d4c2 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:15:34.607524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:34.674328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32deb2d5-4234-4834-9369-f4dd70e20272 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:15:34.674855 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deb2d5-4234-4834-9369-f4dd70e20272 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:34.675139 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deb2d5-4234-4834-9369-f4dd70e20272 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:34.675409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32deb2d5-4234-4834-9369-f4dd70e20272 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:34.677777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-32deb2d5-4234-4834-9369-f4dd70e20272 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:34.773047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32deb2d5-4234-4834-9369-f4dd70e20272 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:15:34.774759 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32deb2d5-4234-4834-9369-f4dd70e20272 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:15:34.794553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 682/1376] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:15:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:15:37.143723 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28246f-b3a5-4bfd-8577-c7bf73eeed6e None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:37.144727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28246f-b3a5-4bfd-8577-c7bf73eeed6e None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:37.145020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28246f-b3a5-4bfd-8577-c7bf73eeed6e None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:37.147126 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 695/1377] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:15:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:37.152684 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000e2e9-aaaf-4b24-87eb-ba046a399e65 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:37.153120 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000e2e9-aaaf-4b24-87eb-ba046a399e65 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:37.153274 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000e2e9-aaaf-4b24-87eb-ba046a399e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:37.231556 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-a000e2e9-aaaf-4b24-87eb-ba046a399e65 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:37.232766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-a000e2e9-aaaf-4b24-87eb-ba046a399e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:37.241119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:37.341765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 683/1378] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:37.631468 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28246f-b3a5-4bfd-8577-c7bf73eeed6e None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:15:37.633111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-bb28246f-b3a5-4bfd-8577-c7bf73eeed6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:37.698201 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:37.795720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28246f-b3a5-4bfd-8577-c7bf73eeed6e None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:15:37.918061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c2fcb-aafc-4844-b68e-ba3a46a7ac8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:15:37.919023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c2fcb-aafc-4844-b68e-ba3a46a7ac8e service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:37.919344 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c2fcb-aafc-4844-b68e-ba3a46a7ac8e service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:37.919687 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c2fcb-aafc-4844-b68e-ba3a46a7ac8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:37.921388 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-c22c2fcb-aafc-4844-b68e-ba3a46a7ac8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:37.943001 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:38.021426 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c2fcb-aafc-4844-b68e-ba3a46a7ac8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:15:38.023131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c2fcb-aafc-4844-b68e-ba3a46a7ac8e service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:15:38.043927 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 696/1379] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:15:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:15:40.260233 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52efe3-aaa3-42ec-83b4-7da84711347b None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:40.260676 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52efe3-aaa3-42ec-83b4-7da84711347b None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:40.260764 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52efe3-aaa3-42ec-83b4-7da84711347b None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:40.262269 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 684/1380] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:40.267484 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5596b1e-8c1b-4148-bdec-383dd65ca385 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:40.267896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5596b1e-8c1b-4148-bdec-383dd65ca385 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:40.268060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5596b1e-8c1b-4148-bdec-383dd65ca385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:40.360615 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-e5596b1e-8c1b-4148-bdec-383dd65ca385 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:40.361942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e5596b1e-8c1b-4148-bdec-383dd65ca385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:40.468594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:40.475494 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 697/1381] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:40.769292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52efe3-aaa3-42ec-83b4-7da84711347b None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:15:40.771694 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-ba52efe3-aaa3-42ec-83b4-7da84711347b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:40.915349 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:40.927064 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52efe3-aaa3-42ec-83b4-7da84711347b None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:15:41.040813 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c381c0-709f-4c91-91ca-116e28eac22c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:15:41.041439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c381c0-709f-4c91-91ca-116e28eac22c service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:41.041721 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c381c0-709f-4c91-91ca-116e28eac22c service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:41.041977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c381c0-709f-4c91-91ca-116e28eac22c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:41.043586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-14c381c0-709f-4c91-91ca-116e28eac22c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:41.143211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c381c0-709f-4c91-91ca-116e28eac22c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:15:41.146127 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c381c0-709f-4c91-91ca-116e28eac22c service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:15:41.159426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:41.173165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 685/1382] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:15:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:15:43.356359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864abfe-b668-41c9-859e-55fd15d2d526 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:43.356916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864abfe-b668-41c9-859e-55fd15d2d526 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:43.356916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864abfe-b668-41c9-859e-55fd15d2d526 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:43.358214 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 698/1383] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:43.364678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87046ff3-0faa-4615-894b-ca7fb397dc3c None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:43.365117 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87046ff3-0faa-4615-894b-ca7fb397dc3c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:43.365375 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87046ff3-0faa-4615-894b-ca7fb397dc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:43.439822 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-87046ff3-0faa-4615-894b-ca7fb397dc3c None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:43.441067 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-87046ff3-0faa-4615-894b-ca7fb397dc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:43.540927 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 686/1384] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:43.831839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864abfe-b668-41c9-859e-55fd15d2d526 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:15:43.833865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-3864abfe-b668-41c9-859e-55fd15d2d526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:43.995436 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864abfe-b668-41c9-859e-55fd15d2d526 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:15:44.067670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:44.115630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d58016-89a8-4e78-b7a4-0be7588feb7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:15:44.116258 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d58016-89a8-4e78-b7a4-0be7588feb7b service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:44.116547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d58016-89a8-4e78-b7a4-0be7588feb7b service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:44.116827 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d58016-89a8-4e78-b7a4-0be7588feb7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:44.118505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-21d58016-89a8-4e78-b7a4-0be7588feb7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:44.214864 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d58016-89a8-4e78-b7a4-0be7588feb7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:15:44.216461 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d58016-89a8-4e78-b7a4-0be7588feb7b service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:15:44.234211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 699/1385] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:15:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:15:46.781766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38878941-b349-4510-b211-54f4c0a92271 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:46.782317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38878941-b349-4510-b211-54f4c0a92271 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:46.782317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38878941-b349-4510-b211-54f4c0a92271 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:46.783721 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 687/1386] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:46.789282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30541100-ba84-443a-848c-64ec3cfb4bd5 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:46.789734 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30541100-ba84-443a-848c-64ec3cfb4bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:46.789831 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30541100-ba84-443a-848c-64ec3cfb4bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:46.857315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-30541100-ba84-443a-848c-64ec3cfb4bd5 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:46.858523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-30541100-ba84-443a-848c-64ec3cfb4bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:46.893056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:46.960140 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 700/1387] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:47.255374 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38878941-b349-4510-b211-54f4c0a92271 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:15:47.257169 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-38878941-b349-4510-b211-54f4c0a92271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:47.338944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:47.410613 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38878941-b349-4510-b211-54f4c0a92271 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:15:47.525657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763da4b-ea09-4b88-92a6-bf0f0d529197 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:15:47.526273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763da4b-ea09-4b88-92a6-bf0f0d529197 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:47.526552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763da4b-ea09-4b88-92a6-bf0f0d529197 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:47.526825 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763da4b-ea09-4b88-92a6-bf0f0d529197 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:47.528430 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-4763da4b-ea09-4b88-92a6-bf0f0d529197 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:47.568263 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:47.624771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763da4b-ea09-4b88-92a6-bf0f0d529197 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:15:47.626450 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763da4b-ea09-4b88-92a6-bf0f0d529197 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:15:47.648055 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 688/1388] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:15:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:15:49.877394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bbf14-4cca-4fd9-a054-1b7c7381c21a None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:49.877924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bbf14-4cca-4fd9-a054-1b7c7381c21a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:49.877924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bbf14-4cca-4fd9-a054-1b7c7381c21a None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:49.879897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 701/1389] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:49.885216 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cea3a-cf46-4547-b7b9-58851ca0ddad None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:49.885635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cea3a-cf46-4547-b7b9-58851ca0ddad None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:49.885748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cea3a-cf46-4547-b7b9-58851ca0ddad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:49.950900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-955cea3a-cf46-4547-b7b9-58851ca0ddad None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:49.952088 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-955cea3a-cf46-4547-b7b9-58851ca0ddad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:50.049110 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 689/1390] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:50.343150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bbf14-4cca-4fd9-a054-1b7c7381c21a None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:15:50.344730 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-bd8bbf14-4cca-4fd9-a054-1b7c7381c21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:50.491450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bbf14-4cca-4fd9-a054-1b7c7381c21a None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:15:50.509049 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:50.611041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572702d-f617-4728-854c-442632d093fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:15:50.611759 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572702d-f617-4728-854c-442632d093fe service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:50.612034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572702d-f617-4728-854c-442632d093fe service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:50.612304 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572702d-f617-4728-854c-442632d093fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:50.614086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7572702d-f617-4728-854c-442632d093fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:50.708081 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572702d-f617-4728-854c-442632d093fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:15:50.709684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572702d-f617-4728-854c-442632d093fe service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:15:50.727367 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 702/1391] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:15:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:15:51.026642 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4a163-9e17-421e-b145-25c78cc67d0f None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:51.027215 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4a163-9e17-421e-b145-25c78cc67d0f None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:51.027215 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4a163-9e17-421e-b145-25c78cc67d0f None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:51.028683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 690/1392] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:51.033874 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a7e36-110a-49e2-9500-0e359a0f5196 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:51.034333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a7e36-110a-49e2-9500-0e359a0f5196 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:51.034447 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a7e36-110a-49e2-9500-0e359a0f5196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:51.060796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:51.103536 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-cb3a7e36-110a-49e2-9500-0e359a0f5196 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:51.104727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-cb3a7e36-110a-49e2-9500-0e359a0f5196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:51.211514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 703/1393] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:15:51.524487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ebc54-f44c-4731-956d-9d0cb0903044 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:51.524806 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ebc54-f44c-4731-956d-9d0cb0903044 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:51.525154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ebc54-f44c-4731-956d-9d0cb0903044 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:51.526572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 691/1394] 10.222.0.60 () {54 vars in 962 bytes} [Wed Sep 28 12:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:51.532367 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372cc7f-ea2b-4139-b7bc-866853147b14 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:51.532655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372cc7f-ea2b-4139-b7bc-866853147b14 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:51.532928 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372cc7f-ea2b-4139-b7bc-866853147b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:51.608591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-b372cc7f-ea2b-4139-b7bc-866853147b14 None None] MFA Rules not processed for user `8e08e00bcb5f47ffab58974ba376938a`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:51.610106 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b372cc7f-ea2b-4139-b7bc-866853147b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:51.662815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:51.710324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 704/1395] 10.222.0.60 () {58 vars in 1059 bytes} [Wed Sep 28 12:15:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:51.722693 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ebc54-f44c-4731-956d-9d0cb0903044 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:15:51.725084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b93ebc54-f44c-4731-956d-9d0cb0903044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:51.739494 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:51.835299 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ebc54-f44c-4731-956d-9d0cb0903044 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:15:51.913791 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c517b0-4df4-487f-a908-3c8b681ed014 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e08e00bcb5f47ffab58974ba376938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:15:51.914404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c517b0-4df4-487f-a908-3c8b681ed014 service glance] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:51.914691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c517b0-4df4-487f-a908-3c8b681ed014 service glance] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:51.914963 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c517b0-4df4-487f-a908-3c8b681ed014 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:51.916563 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-53c517b0-4df4-487f-a908-3c8b681ed014 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:51.995857 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:52.040735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c517b0-4df4-487f-a908-3c8b681ed014 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:15:52.042203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c517b0-4df4-487f-a908-3c8b681ed014 service glance] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:15:52.060820 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 692/1396] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:15:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:15:52.822525 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5a5a0-758c-44f7-b5ad-45e3dc52d8c5 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:52.823124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5a5a0-758c-44f7-b5ad-45e3dc52d8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:52.823474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5a5a0-758c-44f7-b5ad-45e3dc52d8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:52.896334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-d5d5a5a0-758c-44f7-b5ad-45e3dc52d8c5 None None] MFA Rules not processed for user `bc69cf267bea41bf890bc7551d84ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:52.897541 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d5d5a5a0-758c-44f7-b5ad-45e3dc52d8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:52.948992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d67e3-fa03-4f1f-afd6-2fa100b27eb2 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:52.949293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d67e3-fa03-4f1f-afd6-2fa100b27eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:52.949569 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d67e3-fa03-4f1f-afd6-2fa100b27eb2 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:52.950909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 693/1397] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:52.955774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fead72d-6279-4531-9118-7c8bbd869fa3 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:52.956059 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fead72d-6279-4531-9118-7c8bbd869fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:52.956333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fead72d-6279-4531-9118-7c8bbd869fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:52.987078 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 705/1398] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 12:15:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:53.027251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-2fead72d-6279-4531-9118-7c8bbd869fa3 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:53.028564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2fead72d-6279-4531-9118-7c8bbd869fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:53.034550 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15507d52-2f23-48c0-83fb-09368d15818a None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:53.034837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15507d52-2f23-48c0-83fb-09368d15818a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:53.035138 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15507d52-2f23-48c0-83fb-09368d15818a None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:53.037033 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 706/1399] 10.222.0.60 () {54 vars in 943 bytes} [Wed Sep 28 12:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 12:15:53.044015 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1ee43-7d4f-421b-a724-d0906cf763bb None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:53.044301 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1ee43-7d4f-421b-a724-d0906cf763bb None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:53.044572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1ee43-7d4f-421b-a724-d0906cf763bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:53.115449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:53.123208 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-cad1ee43-7d4f-421b-a724-d0906cf763bb None None] MFA Rules not processed for user `975a8d7349994520bcbfb5f75e52aac7`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:53.124415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-cad1ee43-7d4f-421b-a724-d0906cf763bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:53.137006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 694/1400] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:15:53.242853 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 707/1401] 10.222.0.60 () {58 vars in 1040 bytes} [Wed Sep 28 12:15:53 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:15:53.252184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fead72d-6279-4531-9118-7c8bbd869fa3 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:15:53.253492 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2fead72d-6279-4531-9118-7c8bbd869fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:53.280283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:53.358375 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fead72d-6279-4531-9118-7c8bbd869fa3 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:15:53.431326 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1ee43-7d4f-421b-a724-d0906cf763bb None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:15:53.432641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-cad1ee43-7d4f-421b-a724-d0906cf763bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:53.478409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02afe6-6d77-41c4-aca3-9e2beeee82b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975a8d7349994520bcbfb5f75e52aac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:15:53.479045 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02afe6-6d77-41c4-aca3-9e2beeee82b0 service swift] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:53.479369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02afe6-6d77-41c4-aca3-9e2beeee82b0 service swift] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:53.479638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02afe6-6d77-41c4-aca3-9e2beeee82b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:53.481512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-4d02afe6-6d77-41c4-aca3-9e2beeee82b0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:53.526009 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:53.536834 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1ee43-7d4f-421b-a724-d0906cf763bb None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:15:53.558497 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:53.570356 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02afe6-6d77-41c4-aca3-9e2beeee82b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:15:53.571983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02afe6-6d77-41c4-aca3-9e2beeee82b0 service swift] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:15:53.580949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 695/1402] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 12:15:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 12:15:53.661234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16920b-09bb-4032-8471-7689ef8ef5d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:15:53.662007 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16920b-09bb-4032-8471-7689ef8ef5d3 service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:53.662283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16920b-09bb-4032-8471-7689ef8ef5d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:53.662552 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16920b-09bb-4032-8471-7689ef8ef5d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:53.664149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7f16920b-09bb-4032-8471-7689ef8ef5d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:53.793254 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16920b-09bb-4032-8471-7689ef8ef5d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:15:53.794757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16920b-09bb-4032-8471-7689ef8ef5d3 service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:15:53.818436 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 708/1403] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:15:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:15:56.074918 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa9cd0-48e6-4c9f-a004-b9b16ca13bee None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:56.075548 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa9cd0-48e6-4c9f-a004-b9b16ca13bee None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:56.075548 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa9cd0-48e6-4c9f-a004-b9b16ca13bee None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:56.077144 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 696/1404] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:56.082939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cda04-e020-4d14-85d3-1bd8cf5dd85e None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:56.083712 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cda04-e020-4d14-85d3-1bd8cf5dd85e None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:56.083712 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cda04-e020-4d14-85d3-1bd8cf5dd85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:56.120799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:56.160643 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-f78cda04-e020-4d14-85d3-1bd8cf5dd85e None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:56.162327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-f78cda04-e020-4d14-85d3-1bd8cf5dd85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:56.282046 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 709/1405] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:56.586565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa9cd0-48e6-4c9f-a004-b9b16ca13bee None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:15:56.588097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c9fa9cd0-48e6-4c9f-a004-b9b16ca13bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:56.759527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa9cd0-48e6-4c9f-a004-b9b16ca13bee None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:15:56.803577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:56.896217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4f0fe-84c3-41de-b24d-d42091f4d509 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:15:56.896739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4f0fe-84c3-41de-b24d-d42091f4d509 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:56.897014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4f0fe-84c3-41de-b24d-d42091f4d509 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:56.897367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4f0fe-84c3-41de-b24d-d42091f4d509 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:56.898922 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1cf4f0fe-84c3-41de-b24d-d42091f4d509 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:57.009414 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4f0fe-84c3-41de-b24d-d42091f4d509 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:15:57.011276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4f0fe-84c3-41de-b24d-d42091f4d509 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:15:57.031665 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 697/1406] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:15:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:15:59.305620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5b679-40b2-499d-a37e-9ddb3df94282 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:59.306217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5b679-40b2-499d-a37e-9ddb3df94282 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:59.306217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5b679-40b2-499d-a37e-9ddb3df94282 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:59.307506 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 710/1407] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:15:59.312837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5be3e8-f192-4e62-9fb3-bcc762a4e7ee None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:15:59.313293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5be3e8-f192-4e62-9fb3-bcc762a4e7ee None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:15:59.313424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5be3e8-f192-4e62-9fb3-bcc762a4e7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:15:59.365835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:59.386992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-4f5be3e8-f192-4e62-9fb3-bcc762a4e7ee None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:15:59.388234 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-4f5be3e8-f192-4e62-9fb3-bcc762a4e7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:59.495087 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 698/1408] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:15:59.786295 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5b679-40b2-499d-a37e-9ddb3df94282 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:15:59.787843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-93f5b679-40b2-499d-a37e-9ddb3df94282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:15:59.902845 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:15:59.960095 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5b679-40b2-499d-a37e-9ddb3df94282 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:00.085272 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101227f-e820-4d2e-9350-deb514d49302 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:00.086121 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101227f-e820-4d2e-9350-deb514d49302 service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:00.086565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101227f-e820-4d2e-9350-deb514d49302 service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:00.087003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101227f-e820-4d2e-9350-deb514d49302 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:00.088739 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d101227f-e820-4d2e-9350-deb514d49302 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:00.163709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:00.192789 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101227f-e820-4d2e-9350-deb514d49302 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:00.194432 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d101227f-e820-4d2e-9350-deb514d49302 service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:00.216231 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 711/1409] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:16:09.493083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=47 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:10.211668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=16 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:16.295571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afec34-dde4-4c19-a537-d371c8691723 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:16.296116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afec34-dde4-4c19-a537-d371c8691723 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:16.296228 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afec34-dde4-4c19-a537-d371c8691723 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:16.297557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 699/1410] 10.222.0.60 () {54 vars in 918 bytes} [Wed Sep 28 12:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:16.303100 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f183842-c46d-4c0b-a9fc-884b7eaf91c3 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:16.303538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f183842-c46d-4c0b-a9fc-884b7eaf91c3 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:16.303669 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f183842-c46d-4c0b-a9fc-884b7eaf91c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:16.381010 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7f183842-c46d-4c0b-a9fc-884b7eaf91c3 None None] MFA Rules not processed for user `9a2322b1a76d40e49545f5677e247489`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:16.382260 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7f183842-c46d-4c0b-a9fc-884b7eaf91c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:16.488172 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 712/1411] 10.222.0.60 () {58 vars in 1015 bytes} [Wed Sep 28 12:16:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:16.502545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac666f-1a70-4590-a391-939a132b661b None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:16.502836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac666f-1a70-4590-a391-939a132b661b None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:16.503117 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac666f-1a70-4590-a391-939a132b661b None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:16.504446 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 700/1412] 10.222.0.60 () {54 vars in 918 bytes} [Wed Sep 28 12:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:16.509111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9cbcc-0f7e-4985-bb73-54ffaea9656e None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:16.509430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9cbcc-0f7e-4985-bb73-54ffaea9656e None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:16.509703 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9cbcc-0f7e-4985-bb73-54ffaea9656e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:16.581412 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:16.584393 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-dde9cbcc-0f7e-4985-bb73-54ffaea9656e None None] MFA Rules not processed for user `982b82e8ac424318a934f9149fb91706`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:16.586232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-dde9cbcc-0f7e-4985-bb73-54ffaea9656e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:16.700466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 713/1413] 10.222.0.60 () {58 vars in 1015 bytes} [Wed Sep 28 12:16:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:19.252180 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5304bdb-ea07-4a66-b31e-842e57f6dcc1 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:19.252963 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5304bdb-ea07-4a66-b31e-842e57f6dcc1 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:19.252963 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5304bdb-ea07-4a66-b31e-842e57f6dcc1 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:19.254131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 12:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:19.259465 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a2a02-65e9-4653-8529-ce7b2d07c723 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:19.259917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a2a02-65e9-4653-8529-ce7b2d07c723 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:19.260021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a2a02-65e9-4653-8529-ce7b2d07c723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:19.324741 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7b5a2a02-65e9-4653-8529-ce7b2d07c723 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:19.326069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7b5a2a02-65e9-4653-8529-ce7b2d07c723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:19.380805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:19.422893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 12:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:16:19.717198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5304bdb-ea07-4a66-b31e-842e57f6dcc1 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:16:19.718746 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b5304bdb-ea07-4a66-b31e-842e57f6dcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:19.865486 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5304bdb-ea07-4a66-b31e-842e57f6dcc1 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:19.937061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:19.977620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d80404-bf64-43db-a656-ceee003f5671 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:19.978160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d80404-bf64-43db-a656-ceee003f5671 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:19.978442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d80404-bf64-43db-a656-ceee003f5671 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:19.978715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d80404-bf64-43db-a656-ceee003f5671 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:19.980307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-85d80404-bf64-43db-a656-ceee003f5671 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:20.071327 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d80404-bf64-43db-a656-ceee003f5671 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:20.072939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d80404-bf64-43db-a656-ceee003f5671 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:20.090815 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 702/1416] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:16:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:16:21.237067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cee203-ece0-4d01-b6c3-4e517d045c85 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:21.237591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cee203-ece0-4d01-b6c3-4e517d045c85 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:21.237714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cee203-ece0-4d01-b6c3-4e517d045c85 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:21.238980 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 12:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:21.245503 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca767fc4-1510-4b08-aa43-c3956b060c4a None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:21.245932 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca767fc4-1510-4b08-aa43-c3956b060c4a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:21.246056 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca767fc4-1510-4b08-aa43-c3956b060c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:21.322868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-ca767fc4-1510-4b08-aa43-c3956b060c4a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:21.324125 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-ca767fc4-1510-4b08-aa43-c3956b060c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:21.350710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:21.423828 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 12:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:21.575701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cee203-ece0-4d01-b6c3-4e517d045c85 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:16:21.577310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-c3cee203-ece0-4d01-b6c3-4e517d045c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:21.723278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cee203-ece0-4d01-b6c3-4e517d045c85 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:21.763575 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:21.837408 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97da08db-830f-4c4d-b9f8-273f641096b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:21.838016 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da08db-830f-4c4d-b9f8-273f641096b0 service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:21.838289 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da08db-830f-4c4d-b9f8-273f641096b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:21.838553 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97da08db-830f-4c4d-b9f8-273f641096b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:21.840119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-97da08db-830f-4c4d-b9f8-273f641096b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:21.927849 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97da08db-830f-4c4d-b9f8-273f641096b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:21.929471 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97da08db-830f-4c4d-b9f8-273f641096b0 service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:21.947671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 716/1419] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:16:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:16:23.098174 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49c145-a55f-4131-94fb-e4d72dd6f8eb None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:23.098752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49c145-a55f-4131-94fb-e4d72dd6f8eb None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:23.098752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49c145-a55f-4131-94fb-e4d72dd6f8eb None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:23.100050 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 12:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:23.106640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65afcb-18cd-4056-bc57-af5ca4fe0659 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:23.107069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65afcb-18cd-4056-bc57-af5ca4fe0659 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:23.107187 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb65afcb-18cd-4056-bc57-af5ca4fe0659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:23.161566 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:23.185127 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-cb65afcb-18cd-4056-bc57-af5ca4fe0659 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:23.186366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-cb65afcb-18cd-4056-bc57-af5ca4fe0659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:23.282475 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 12:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:23.434531 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49c145-a55f-4131-94fb-e4d72dd6f8eb None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:16:23.436032 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0a49c145-a55f-4131-94fb-e4d72dd6f8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:23.586165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49c145-a55f-4131-94fb-e4d72dd6f8eb None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:23.598047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:23.700570 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac64af1-5acf-46e7-ac2c-96a7b265384a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:23.701267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac64af1-5acf-46e7-ac2c-96a7b265384a service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:23.701551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac64af1-5acf-46e7-ac2c-96a7b265384a service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:23.701822 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac64af1-5acf-46e7-ac2c-96a7b265384a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:23.703449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1ac64af1-5acf-46e7-ac2c-96a7b265384a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:23.792114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac64af1-5acf-46e7-ac2c-96a7b265384a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:23.793748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac64af1-5acf-46e7-ac2c-96a7b265384a service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:23.811961 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 705/1422] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:16:24.932823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e507bf7-cb6b-494c-99bf-2035db5b84d2 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:24.933435 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e507bf7-cb6b-494c-99bf-2035db5b84d2 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:24.933435 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e507bf7-cb6b-494c-99bf-2035db5b84d2 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:24.934957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 12:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:24.941712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11386829-708f-4bc9-81d1-c1d88ea54d74 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:24.942175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11386829-708f-4bc9-81d1-c1d88ea54d74 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:24.942281 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11386829-708f-4bc9-81d1-c1d88ea54d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:24.962295 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:25.019077 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-11386829-708f-4bc9-81d1-c1d88ea54d74 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:25.020264 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-11386829-708f-4bc9-81d1-c1d88ea54d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:25.116058 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 12:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:25.269001 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e507bf7-cb6b-494c-99bf-2035db5b84d2 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:16:25.270606 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7e507bf7-cb6b-494c-99bf-2035db5b84d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:25.379896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:25.415605 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e507bf7-cb6b-494c-99bf-2035db5b84d2 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:25.526697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9a45c-8d13-45a6-958b-e35d131538fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:25.527215 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9a45c-8d13-45a6-958b-e35d131538fb service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:25.527490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9a45c-8d13-45a6-958b-e35d131538fb service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:25.527757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9a45c-8d13-45a6-958b-e35d131538fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:25.529342 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-c6d9a45c-8d13-45a6-958b-e35d131538fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:25.604849 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:25.619495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9a45c-8d13-45a6-958b-e35d131538fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:25.621101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9a45c-8d13-45a6-958b-e35d131538fb service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:25.638773 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 719/1425] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:16:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:16:26.742381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b3dbc-02c1-4705-b2af-ac0d2bf4d9f0 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:26.742953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b3dbc-02c1-4705-b2af-ac0d2bf4d9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:26.742953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b3dbc-02c1-4705-b2af-ac0d2bf4d9f0 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:26.744285 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 12:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:26.750536 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577836ab-8352-451c-bc9b-640651362db1 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:26.751004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577836ab-8352-451c-bc9b-640651362db1 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:26.751108 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577836ab-8352-451c-bc9b-640651362db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:26.826378 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-577836ab-8352-451c-bc9b-640651362db1 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:26.827572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-577836ab-8352-451c-bc9b-640651362db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:26.921667 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 12:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:27.070235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b3dbc-02c1-4705-b2af-ac0d2bf4d9f0 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:16:27.071657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e44b3dbc-02c1-4705-b2af-ac0d2bf4d9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:27.154936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:27.217048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b3dbc-02c1-4705-b2af-ac0d2bf4d9f0 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:27.331212 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e134aff-092a-489e-a622-ee4653ce5abc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:27.331739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e134aff-092a-489e-a622-ee4653ce5abc service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:27.332016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e134aff-092a-489e-a622-ee4653ce5abc service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:27.332288 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e134aff-092a-489e-a622-ee4653ce5abc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:27.333901 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0e134aff-092a-489e-a622-ee4653ce5abc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:27.383060 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:27.424885 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e134aff-092a-489e-a622-ee4653ce5abc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:27.426516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e134aff-092a-489e-a622-ee4653ce5abc service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:27.445305 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 708/1428] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:16:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:16:28.503169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f015e0d-0a19-4945-84be-038449f66b00 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:28.503748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f015e0d-0a19-4945-84be-038449f66b00 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:28.503748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f015e0d-0a19-4945-84be-038449f66b00 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:28.505060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 12:16:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:28.511589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d49375-3fca-4d14-b199-4c731de00a44 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:28.512067 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d49375-3fca-4d14-b199-4c731de00a44 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:28.512173 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d49375-3fca-4d14-b199-4c731de00a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:28.588070 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-83d49375-3fca-4d14-b199-4c731de00a44 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:28.589334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-83d49375-3fca-4d14-b199-4c731de00a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:28.685636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 12:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:28.836244 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f015e0d-0a19-4945-84be-038449f66b00 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:16:28.837708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9f015e0d-0a19-4945-84be-038449f66b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:28.886155 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:28.983533 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f015e0d-0a19-4945-84be-038449f66b00 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:29.094218 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fabc0-2b8f-421a-9ab0-1a159d753280 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:29.094741 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fabc0-2b8f-421a-9ab0-1a159d753280 service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:29.095015 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fabc0-2b8f-421a-9ab0-1a159d753280 service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:29.095923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fabc0-2b8f-421a-9ab0-1a159d753280 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:29.097526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-0f0fabc0-2b8f-421a-9ab0-1a159d753280 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:29.104290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:29.188683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fabc0-2b8f-421a-9ab0-1a159d753280 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:29.190802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fabc0-2b8f-421a-9ab0-1a159d753280 service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:29.209197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 722/1431] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:16:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:16:30.299023 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931cfb4-5674-4cef-aae9-3c6991aaea62 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:30.299603 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931cfb4-5674-4cef-aae9-3c6991aaea62 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:30.299708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931cfb4-5674-4cef-aae9-3c6991aaea62 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:30.301041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 12:16:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:30.307287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8a6c0-b5bd-4181-a95c-4ef5f2196109 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:30.307714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8a6c0-b5bd-4181-a95c-4ef5f2196109 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:30.307828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8a6c0-b5bd-4181-a95c-4ef5f2196109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:30.383486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-90a8a6c0-b5bd-4181-a95c-4ef5f2196109 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:30.384690 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-90a8a6c0-b5bd-4181-a95c-4ef5f2196109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:30.452375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:30.486716 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 12:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:30.639196 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931cfb4-5674-4cef-aae9-3c6991aaea62 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:16:30.640700 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f931cfb4-5674-4cef-aae9-3c6991aaea62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:30.659504 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:30.787273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931cfb4-5674-4cef-aae9-3c6991aaea62 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:30.874598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:30.901113 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb187a-f053-4634-aedd-7cd1b8be062c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:30.901742 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb187a-f053-4634-aedd-7cd1b8be062c service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:30.902019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb187a-f053-4634-aedd-7cd1b8be062c service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:30.902290 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb187a-f053-4634-aedd-7cd1b8be062c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:30.903881 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b2eb187a-f053-4634-aedd-7cd1b8be062c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:30.992011 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb187a-f053-4634-aedd-7cd1b8be062c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:30.993632 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb187a-f053-4634-aedd-7cd1b8be062c service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:31.011806 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 711/1434] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:16:32.074032 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dcd636-8935-4074-9d4d-55e9a665e919 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:32.074565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dcd636-8935-4074-9d4d-55e9a665e919 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:32.074565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dcd636-8935-4074-9d4d-55e9a665e919 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:32.075878 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 12:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:32.082368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222b46b-912b-4e19-b08a-e317c1fd45fe None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:32.082784 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222b46b-912b-4e19-b08a-e317c1fd45fe None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:32.082912 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222b46b-912b-4e19-b08a-e317c1fd45fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:32.158247 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-a222b46b-912b-4e19-b08a-e317c1fd45fe None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:32.159442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-a222b46b-912b-4e19-b08a-e317c1fd45fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:32.200655 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:32.256699 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 12:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:32.408382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dcd636-8935-4074-9d4d-55e9a665e919 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:16:32.409856 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-f2dcd636-8935-4074-9d4d-55e9a665e919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:32.554279 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dcd636-8935-4074-9d4d-55e9a665e919 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:32.608470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:32.666984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31b979-9f51-4006-a18f-04731540ea93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:32.667515 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31b979-9f51-4006-a18f-04731540ea93 service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:32.667796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31b979-9f51-4006-a18f-04731540ea93 service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:32.668064 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31b979-9f51-4006-a18f-04731540ea93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:32.669718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-0f31b979-9f51-4006-a18f-04731540ea93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:32.761823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31b979-9f51-4006-a18f-04731540ea93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:32.763414 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31b979-9f51-4006-a18f-04731540ea93 service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:32.780836 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 725/1437] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:16:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:16:33.838092 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b07b52-61f1-40f6-b260-9b346ed97290 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:33.838584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b07b52-61f1-40f6-b260-9b346ed97290 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:33.838689 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b07b52-61f1-40f6-b260-9b346ed97290 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:33.839969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 12:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:33.847153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3430dbd-b0c8-4dc8-8eae-31cfc4e8a9f1 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:33.847631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3430dbd-b0c8-4dc8-8eae-31cfc4e8a9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:33.847727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3430dbd-b0c8-4dc8-8eae-31cfc4e8a9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:33.921882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-d3430dbd-b0c8-4dc8-8eae-31cfc4e8a9f1 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:33.923116 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d3430dbd-b0c8-4dc8-8eae-31cfc4e8a9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:33.925632 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:34.019346 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 12:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:34.170028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b07b52-61f1-40f6-b260-9b346ed97290 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:16:34.171885 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d8b07b52-61f1-40f6-b260-9b346ed97290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:34.319329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b07b52-61f1-40f6-b260-9b346ed97290 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:34.347772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:34.431742 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef03bb-0e4d-45de-ac0b-d78930afa456 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:34.432343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef03bb-0e4d-45de-ac0b-d78930afa456 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:34.432619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef03bb-0e4d-45de-ac0b-d78930afa456 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:34.432886 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef03bb-0e4d-45de-ac0b-d78930afa456 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:34.434497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8eef03bb-0e4d-45de-ac0b-d78930afa456 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:34.523310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef03bb-0e4d-45de-ac0b-d78930afa456 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:34.524904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef03bb-0e4d-45de-ac0b-d78930afa456 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:34.542650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 714/1440] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:16:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:16:36.588871 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb54d2-5106-4693-b9fc-cdca1efe6776 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:36.589448 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb54d2-5106-4693-b9fc-cdca1efe6776 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:36.589448 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb54d2-5106-4693-b9fc-cdca1efe6776 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:36.590770 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 12:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:36.595997 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679758e4-fee3-4737-8b1a-86e3f10afb63 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:36.596454 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679758e4-fee3-4737-8b1a-86e3f10afb63 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:36.596560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679758e4-fee3-4737-8b1a-86e3f10afb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:36.638010 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:36.664631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-679758e4-fee3-4737-8b1a-86e3f10afb63 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:36.665863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-679758e4-fee3-4737-8b1a-86e3f10afb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:36.763536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 12:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:37.048356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb54d2-5106-4693-b9fc-cdca1efe6776 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:16:37.049928 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-cebb54d2-5106-4693-b9fc-cdca1efe6776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:37.152516 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb54d2-5106-4693-b9fc-cdca1efe6776 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:37.185686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:37.228534 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010fbb6-a58c-483c-8b46-e8baebd15459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:37.229158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010fbb6-a58c-483c-8b46-e8baebd15459 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:37.229444 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010fbb6-a58c-483c-8b46-e8baebd15459 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:37.229719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010fbb6-a58c-483c-8b46-e8baebd15459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:37.231293 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-c010fbb6-a58c-483c-8b46-e8baebd15459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:37.341031 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010fbb6-a58c-483c-8b46-e8baebd15459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:37.342412 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010fbb6-a58c-483c-8b46-e8baebd15459 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:37.480467 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 728/1443] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:16:39.840078 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d102a-a670-4c0d-a3aa-e6148849751e None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:39.840517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d102a-a670-4c0d-a3aa-e6148849751e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:39.840635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d102a-a670-4c0d-a3aa-e6148849751e None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:39.842032 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 12:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:39.847581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beaed11-ffab-49be-9241-dd37964862b8 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:39.848003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beaed11-ffab-49be-9241-dd37964862b8 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:39.848114 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beaed11-ffab-49be-9241-dd37964862b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:39.913059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-3beaed11-ffab-49be-9241-dd37964862b8 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:39.914277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-3beaed11-ffab-49be-9241-dd37964862b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:39.926869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:40.010051 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 12:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:40.294618 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d102a-a670-4c0d-a3aa-e6148849751e None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:16:40.296170 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-cd1d102a-a670-4c0d-a3aa-e6148849751e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:40.399180 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d102a-a670-4c0d-a3aa-e6148849751e None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:40.412655 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:40.472848 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8190968-e234-4d4c-9bdc-a73f98201c69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:40.473484 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8190968-e234-4d4c-9bdc-a73f98201c69 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:40.473758 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8190968-e234-4d4c-9bdc-a73f98201c69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:40.474027 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8190968-e234-4d4c-9bdc-a73f98201c69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:40.475600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c8190968-e234-4d4c-9bdc-a73f98201c69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:40.583420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8190968-e234-4d4c-9bdc-a73f98201c69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:40.584897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8190968-e234-4d4c-9bdc-a73f98201c69 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:40.603851 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 717/1446] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:16:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:16:43.052949 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b3823-ae44-409f-b766-f321a1045775 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:43.053552 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b3823-ae44-409f-b766-f321a1045775 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:43.053552 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b3823-ae44-409f-b766-f321a1045775 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:43.054902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 12:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:43.060308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc3b0a-a669-4e17-9209-5682896528c0 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:43.060742 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc3b0a-a669-4e17-9209-5682896528c0 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:43.060838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc3b0a-a669-4e17-9209-5682896528c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:43.111346 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:43.130119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-90bc3b0a-a669-4e17-9209-5682896528c0 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:43.131323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-90bc3b0a-a669-4e17-9209-5682896528c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:43.227805 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 12:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:43.524351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b3823-ae44-409f-b766-f321a1045775 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:16:43.525917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a26b3823-ae44-409f-b766-f321a1045775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:43.628951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b3823-ae44-409f-b766-f321a1045775 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:43.668679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:43.701489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf85dda-0c5e-4b63-8b85-b3b7f9e66eab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:43.702087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf85dda-0c5e-4b63-8b85-b3b7f9e66eab service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:43.702360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf85dda-0c5e-4b63-8b85-b3b7f9e66eab service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:43.702625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf85dda-0c5e-4b63-8b85-b3b7f9e66eab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:43.704200 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9cf85dda-0c5e-4b63-8b85-b3b7f9e66eab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:43.815691 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf85dda-0c5e-4b63-8b85-b3b7f9e66eab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:43.817013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf85dda-0c5e-4b63-8b85-b3b7f9e66eab service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:43.836283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 731/1449] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:16:46.205170 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5938afe-b3c2-4fbc-8575-367ee7d40886 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:46.205708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5938afe-b3c2-4fbc-8575-367ee7d40886 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:46.205803 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5938afe-b3c2-4fbc-8575-367ee7d40886 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:46.207370 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 12:16:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:46.212560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eb784-b46a-4b9e-8a07-579b6bfe9650 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:46.212994 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eb784-b46a-4b9e-8a07-579b6bfe9650 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:46.213108 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0eb784-b46a-4b9e-8a07-579b6bfe9650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:46.279289 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-6c0eb784-b46a-4b9e-8a07-579b6bfe9650 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:46.280481 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-6c0eb784-b46a-4b9e-8a07-579b6bfe9650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:46.301891 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:46.381510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 12:16:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:46.675182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5938afe-b3c2-4fbc-8575-367ee7d40886 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:16:46.676744 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f5938afe-b3c2-4fbc-8575-367ee7d40886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:46.780006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5938afe-b3c2-4fbc-8575-367ee7d40886 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:46.801023 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:46.857038 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143616d-dfe8-4594-bbd8-417684e2a065 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:46.857926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143616d-dfe8-4594-bbd8-417684e2a065 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:46.858211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143616d-dfe8-4594-bbd8-417684e2a065 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:46.858481 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143616d-dfe8-4594-bbd8-417684e2a065 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:46.860059 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5143616d-dfe8-4594-bbd8-417684e2a065 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:46.970632 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143616d-dfe8-4594-bbd8-417684e2a065 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:46.972043 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143616d-dfe8-4594-bbd8-417684e2a065 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:46.989947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 720/1452] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:16:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:16:49.285070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892c543-c78e-452a-946e-d9325dd34505 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:49.285657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892c543-c78e-452a-946e-d9325dd34505 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:49.285657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892c543-c78e-452a-946e-d9325dd34505 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:49.286979 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 12:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:49.292339 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914afff-48f1-440d-913f-ed9ee57e6760 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:49.292753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914afff-48f1-440d-913f-ed9ee57e6760 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:49.292881 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914afff-48f1-440d-913f-ed9ee57e6760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:49.357833 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:49.361571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-c914afff-48f1-440d-913f-ed9ee57e6760 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:49.364856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c914afff-48f1-440d-913f-ed9ee57e6760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:49.470946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 12:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:49.757192 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892c543-c78e-452a-946e-d9325dd34505 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:16:49.758713 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e892c543-c78e-452a-946e-d9325dd34505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:49.868770 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892c543-c78e-452a-946e-d9325dd34505 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:49.917783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:49.942592 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b81ec5-77c9-4ffe-a13a-8f876f06fc7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:49.943307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b81ec5-77c9-4ffe-a13a-8f876f06fc7a service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:49.943586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b81ec5-77c9-4ffe-a13a-8f876f06fc7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:49.943854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b81ec5-77c9-4ffe-a13a-8f876f06fc7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:49.945442 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-41b81ec5-77c9-4ffe-a13a-8f876f06fc7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:50.056332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b81ec5-77c9-4ffe-a13a-8f876f06fc7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:50.057708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b81ec5-77c9-4ffe-a13a-8f876f06fc7a service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:50.075841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 734/1455] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:16:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:16:52.301200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b4db5-cc02-43c1-9e6d-01f9d98dde51 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:52.301741 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b4db5-cc02-43c1-9e6d-01f9d98dde51 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:52.301741 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b4db5-cc02-43c1-9e6d-01f9d98dde51 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:52.303141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 12:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:52.308279 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585f004-8547-4384-95bc-e9197fc41193 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:52.308711 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585f004-8547-4384-95bc-e9197fc41193 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:52.308822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7585f004-8547-4384-95bc-e9197fc41193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:52.373725 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7585f004-8547-4384-95bc-e9197fc41193 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:52.374993 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7585f004-8547-4384-95bc-e9197fc41193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:52.403623 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:52.471140 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 12:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:52.784424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b4db5-cc02-43c1-9e6d-01f9d98dde51 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:16:52.786007 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-9f0b4db5-cc02-43c1-9e6d-01f9d98dde51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:52.890633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b4db5-cc02-43c1-9e6d-01f9d98dde51 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:52.920265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:52.966267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726697a-7862-4cc2-a4f5-55ef4e7c721e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:52.966875 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726697a-7862-4cc2-a4f5-55ef4e7c721e service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:52.967198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726697a-7862-4cc2-a4f5-55ef4e7c721e service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:52.967612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726697a-7862-4cc2-a4f5-55ef4e7c721e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:52.969468 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c726697a-7862-4cc2-a4f5-55ef4e7c721e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:53.090846 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726697a-7862-4cc2-a4f5-55ef4e7c721e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:53.092417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726697a-7862-4cc2-a4f5-55ef4e7c721e service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:53.114342 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 723/1458] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:16:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:16:55.455342 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c4c73-4d11-4737-9289-dd425ebb262b None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:55.455992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c4c73-4d11-4737-9289-dd425ebb262b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:55.455992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c4c73-4d11-4737-9289-dd425ebb262b None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:55.457255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 12:16:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:55.462737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dc9b7-db1f-436f-b121-c7238f83256f None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:55.463158 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dc9b7-db1f-436f-b121-c7238f83256f None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:55.463265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dc9b7-db1f-436f-b121-c7238f83256f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:55.530477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-aa4dc9b7-db1f-436f-b121-c7238f83256f None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:55.531707 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-aa4dc9b7-db1f-436f-b121-c7238f83256f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:55.540232 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:55.637831 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 12:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:55.921173 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c4c73-4d11-4737-9289-dd425ebb262b None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:16:55.923002 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-fa2c4c73-4d11-4737-9289-dd425ebb262b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:56.027158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c4c73-4d11-4737-9289-dd425ebb262b None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:56.083064 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:56.104483 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a52b53-1791-4447-8f03-84f84b7ffc3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:56.105093 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a52b53-1791-4447-8f03-84f84b7ffc3d service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:56.105402 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a52b53-1791-4447-8f03-84f84b7ffc3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:56.105670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a52b53-1791-4447-8f03-84f84b7ffc3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:56.107511 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-30a52b53-1791-4447-8f03-84f84b7ffc3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:56.223331 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a52b53-1791-4447-8f03-84f84b7ffc3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:56.224735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a52b53-1791-4447-8f03-84f84b7ffc3d service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:56.244590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 737/1461] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:16:58.472834 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c950a-f55f-4662-b7f0-ac0340a46d10 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:58.473544 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c950a-f55f-4662-b7f0-ac0340a46d10 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:58.473544 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c950a-f55f-4662-b7f0-ac0340a46d10 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:58.474809 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 12:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:16:58.480164 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f19d3-fd14-42c4-b333-2b123de9e716 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:58.480604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f19d3-fd14-42c4-b333-2b123de9e716 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:58.480701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f19d3-fd14-42c4-b333-2b123de9e716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:58.546087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-715f19d3-fd14-42c4-b333-2b123de9e716 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:16:58.547330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-715f19d3-fd14-42c4-b333-2b123de9e716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:58.585292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:58.649889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 12:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:16:58.941185 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c950a-f55f-4662-b7f0-ac0340a46d10 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:16:58.942729 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-9e7c950a-f55f-4662-b7f0-ac0340a46d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:59.046828 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c950a-f55f-4662-b7f0-ac0340a46d10 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:16:59.084948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:16:59.121777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8db40-1853-44fa-a124-62f4c609acf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:16:59.122384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8db40-1853-44fa-a124-62f4c609acf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:16:59.122661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8db40-1853-44fa-a124-62f4c609acf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:16:59.122930 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8db40-1853-44fa-a124-62f4c609acf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:16:59.124524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-75c8db40-1853-44fa-a124-62f4c609acf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:16:59.237598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8db40-1853-44fa-a124-62f4c609acf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:16:59.238958 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8db40-1853-44fa-a124-62f4c609acf3 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:16:59.257528 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 726/1464] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:16:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:17:01.429885 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc0cb2-d32c-43f4-9324-91fcf3ec848d None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:01.430454 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc0cb2-d32c-43f4-9324-91fcf3ec848d None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:01.430454 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc0cb2-d32c-43f4-9324-91fcf3ec848d None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:01.431825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 12:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:17:01.437050 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5777b60-f2f0-4816-a9c5-f7da57c565d3 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:01.437503 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5777b60-f2f0-4816-a9c5-f7da57c565d3 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:01.437617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5777b60-f2f0-4816-a9c5-f7da57c565d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:01.504532 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e5777b60-f2f0-4816-a9c5-f7da57c565d3 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:17:01.505762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e5777b60-f2f0-4816-a9c5-f7da57c565d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:01.523429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:01.605968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 12:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:17:01.912319 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc0cb2-d32c-43f4-9324-91fcf3ec848d None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:17:01.913891 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b2dc0cb2-d32c-43f4-9324-91fcf3ec848d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:02.016453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc0cb2-d32c-43f4-9324-91fcf3ec848d None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:17:02.083553 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:02.090393 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059508d-cb38-45e0-b7f9-498f03523e2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:17:02.091049 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059508d-cb38-45e0-b7f9-498f03523e2a service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:02.091324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059508d-cb38-45e0-b7f9-498f03523e2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:02.091591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059508d-cb38-45e0-b7f9-498f03523e2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:02.093241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-3059508d-cb38-45e0-b7f9-498f03523e2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:02.202647 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059508d-cb38-45e0-b7f9-498f03523e2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:17:02.204687 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059508d-cb38-45e0-b7f9-498f03523e2a service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:17:02.223952 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 740/1467] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:17:04.433115 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897b344-c456-4744-a9bb-72afb55bacc0 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:04.433684 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897b344-c456-4744-a9bb-72afb55bacc0 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:04.433684 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897b344-c456-4744-a9bb-72afb55bacc0 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:04.435065 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 12:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:17:04.441890 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd0ba1-815c-4ad6-b71a-3f6c0396cee3 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:04.442324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd0ba1-815c-4ad6-b71a-3f6c0396cee3 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:04.442570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd0ba1-815c-4ad6-b71a-3f6c0396cee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:04.508672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-d4cd0ba1-815c-4ad6-b71a-3f6c0396cee3 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:17:04.509920 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d4cd0ba1-815c-4ad6-b71a-3f6c0396cee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:04.557698 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:04.606667 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 12:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:17:04.891934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897b344-c456-4744-a9bb-72afb55bacc0 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:17:04.893572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7897b344-c456-4744-a9bb-72afb55bacc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:05.002119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897b344-c456-4744-a9bb-72afb55bacc0 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:17:05.054627 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:05.082161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcaea1-5096-48a2-acf7-56378ceafcbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:17:05.082766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcaea1-5096-48a2-acf7-56378ceafcbc service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:05.083046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcaea1-5096-48a2-acf7-56378ceafcbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:05.083314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcaea1-5096-48a2-acf7-56378ceafcbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:05.084884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-dffcaea1-5096-48a2-acf7-56378ceafcbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:05.196150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffcaea1-5096-48a2-acf7-56378ceafcbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:17:05.197633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffcaea1-5096-48a2-acf7-56378ceafcbc service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:17:05.220087 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 729/1470] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:17:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:17:14.605118 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=22 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:15.216527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=66 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:25.300229 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb36b03-55fc-423f-b516-b37b36539624 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:25.300800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb36b03-55fc-423f-b516-b37b36539624 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:25.300800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb36b03-55fc-423f-b516-b37b36539624 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:25.302135 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 742/1471] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:17:25.307507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4435ff0-986b-4743-97c7-a2788df20b23 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:25.307922 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4435ff0-986b-4743-97c7-a2788df20b23 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:25.308054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4435ff0-986b-4743-97c7-a2788df20b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:25.375619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-a4435ff0-986b-4743-97c7-a2788df20b23 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:17:25.376822 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-a4435ff0-986b-4743-97c7-a2788df20b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:25.474792 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 730/1472] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:17:25.773490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f63b3a-cd5b-4b53-afd8-a216280964e3 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:25.773779 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f63b3a-cd5b-4b53-afd8-a216280964e3 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:25.774053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f63b3a-cd5b-4b53-afd8-a216280964e3 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:25.775376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 743/1473] 10.222.0.60 () {54 vars in 962 bytes} [Wed Sep 28 12:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:17:25.780544 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d99184-c3f4-4296-b4f4-62b58e4d750e None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:25.780827 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d99184-c3f4-4296-b4f4-62b58e4d750e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:25.781099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d99184-c3f4-4296-b4f4-62b58e4d750e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:25.858699 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-76d99184-c3f4-4296-b4f4-62b58e4d750e None None] MFA Rules not processed for user `8e08e00bcb5f47ffab58974ba376938a`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:17:25.859944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-76d99184-c3f4-4296-b4f4-62b58e4d750e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:25.881496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:25.937070 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 731/1474] 10.222.0.60 () {58 vars in 1059 bytes} [Wed Sep 28 12:17:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:17:25.944845 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f63b3a-cd5b-4b53-afd8-a216280964e3 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:17:25.946439 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d8f63b3a-cd5b-4b53-afd8-a216280964e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:26.050015 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f63b3a-cd5b-4b53-afd8-a216280964e3 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:17:26.130537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087630cc-adec-4c88-8a1c-4f263a7c1d2c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e08e00bcb5f47ffab58974ba376938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:17:26.131143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087630cc-adec-4c88-8a1c-4f263a7c1d2c service glance] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:26.131423 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087630cc-adec-4c88-8a1c-4f263a7c1d2c service glance] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:26.131694 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087630cc-adec-4c88-8a1c-4f263a7c1d2c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:26.133473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-087630cc-adec-4c88-8a1c-4f263a7c1d2c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:26.190784 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:26.247820 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087630cc-adec-4c88-8a1c-4f263a7c1d2c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:17:26.249486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087630cc-adec-4c88-8a1c-4f263a7c1d2c service glance] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:17:26.268634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 744/1475] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:17:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:17:28.516101 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2541bf5-3b6a-4e00-b9ff-af6f44e2b271 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:28.516685 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2541bf5-3b6a-4e00-b9ff-af6f44e2b271 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:28.516685 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2541bf5-3b6a-4e00-b9ff-af6f44e2b271 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:28.518053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 732/1476] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:17:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:17:28.523360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b67b0f-203f-476a-a7e0-5fc8f6fa3814 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:28.523804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b67b0f-203f-476a-a7e0-5fc8f6fa3814 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:28.523906 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b67b0f-203f-476a-a7e0-5fc8f6fa3814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:28.588128 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-37b67b0f-203f-476a-a7e0-5fc8f6fa3814 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:17:28.589337 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-37b67b0f-203f-476a-a7e0-5fc8f6fa3814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:28.688872 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 745/1477] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:17:28.966363 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2541bf5-3b6a-4e00-b9ff-af6f44e2b271 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:17:28.968516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f2541bf5-3b6a-4e00-b9ff-af6f44e2b271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:29.094188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2541bf5-3b6a-4e00-b9ff-af6f44e2b271 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:17:29.144503 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:29.189695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e60632-ee61-47be-b65f-1a874302203c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:17:29.190377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e60632-ee61-47be-b65f-1a874302203c admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:29.190661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e60632-ee61-47be-b65f-1a874302203c admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:29.190934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e60632-ee61-47be-b65f-1a874302203c admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:29.191573 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e60632-ee61-47be-b65f-1a874302203c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:17:29.193413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: WARNING py.warnings [None req-f5e60632-ee61-47be-b65f-1a874302203c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 12:17:29.193413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: warnings.warn(msg) Sep 28 12:17:29.193413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: Sep 28 12:17:29.194556 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e60632-ee61-47be-b65f-1a874302203c admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:17:29.214789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 733/1478] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Sep 28 12:17:28 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:17:31.381074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80a8cb-63f1-4e14-b74b-b20903c5d77d None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:31.381671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80a8cb-63f1-4e14-b74b-b20903c5d77d None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:31.381781 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80a8cb-63f1-4e14-b74b-b20903c5d77d None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:31.383098 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 746/1479] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:17:31.388379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f55dc2-0646-4c02-a9dd-958987b65796 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:31.388825 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f55dc2-0646-4c02-a9dd-958987b65796 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:31.389002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f55dc2-0646-4c02-a9dd-958987b65796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:31.454774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-c9f55dc2-0646-4c02-a9dd-958987b65796 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:17:31.455944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c9f55dc2-0646-4c02-a9dd-958987b65796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:31.552584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 734/1480] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:17:31.849355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80a8cb-63f1-4e14-b74b-b20903c5d77d None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:17:31.851189 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2c80a8cb-63f1-4e14-b74b-b20903c5d77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:31.862654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:31.998167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80a8cb-63f1-4e14-b74b-b20903c5d77d None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:17:32.075893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:32.109113 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e882712-2e86-47a5-bc46-68a99cd56ba7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:17:32.109679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e882712-2e86-47a5-bc46-68a99cd56ba7 service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:32.109960 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e882712-2e86-47a5-bc46-68a99cd56ba7 service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:32.110227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e882712-2e86-47a5-bc46-68a99cd56ba7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:32.111800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7e882712-2e86-47a5-bc46-68a99cd56ba7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:32.201641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e882712-2e86-47a5-bc46-68a99cd56ba7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:17:32.203517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e882712-2e86-47a5-bc46-68a99cd56ba7 service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:17:32.222029 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 747/1481] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:17:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:17:34.558477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7223b1d-88b9-4f8f-8520-75931c85904a None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:34.559054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7223b1d-88b9-4f8f-8520-75931c85904a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:34.559054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7223b1d-88b9-4f8f-8520-75931c85904a None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:34.560360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 735/1482] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:17:34.565732 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c36d0-4a47-40df-92e4-2cd1fce13f8c None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:34.566177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c36d0-4a47-40df-92e4-2cd1fce13f8c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:34.566306 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c36d0-4a47-40df-92e4-2cd1fce13f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:34.631438 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-545c36d0-4a47-40df-92e4-2cd1fce13f8c None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:17:34.632642 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-545c36d0-4a47-40df-92e4-2cd1fce13f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:34.678540 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:34.729703 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 748/1483] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:17:35.021977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7223b1d-88b9-4f8f-8520-75931c85904a None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:17:35.023518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d7223b1d-88b9-4f8f-8520-75931c85904a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:35.047278 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:35.126714 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7223b1d-88b9-4f8f-8520-75931c85904a None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:17:35.200693 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a6d9f-9e38-4c6b-b3d7-d358709d42e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:17:35.201355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a6d9f-9e38-4c6b-b3d7-d358709d42e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:35.201632 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a6d9f-9e38-4c6b-b3d7-d358709d42e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:35.201898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a6d9f-9e38-4c6b-b3d7-d358709d42e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:35.203458 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d24a6d9f-9e38-4c6b-b3d7-d358709d42e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:35.283414 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:35.318517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a6d9f-9e38-4c6b-b3d7-d358709d42e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:17:35.319953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a6d9f-9e38-4c6b-b3d7-d358709d42e8 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:17:35.338215 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 736/1484] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:17:37.553684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066897f-c230-4e29-b973-12758eba9a27 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:37.554251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066897f-c230-4e29-b973-12758eba9a27 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:37.554251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8066897f-c230-4e29-b973-12758eba9a27 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:37.555559 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 749/1485] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:17:37.560978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288600a9-d61e-446c-adba-c97ace4a6124 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:37.561417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288600a9-d61e-446c-adba-c97ace4a6124 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:37.561570 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288600a9-d61e-446c-adba-c97ace4a6124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:37.627236 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-288600a9-d61e-446c-adba-c97ace4a6124 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:17:37.628445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-288600a9-d61e-446c-adba-c97ace4a6124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:37.724133 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 737/1486] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:17:38.020591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066897f-c230-4e29-b973-12758eba9a27 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:17:38.022147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-8066897f-c230-4e29-b973-12758eba9a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:38.122689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8066897f-c230-4e29-b973-12758eba9a27 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:17:38.195508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:38.197062 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ed3c8-9ab1-4717-a5e9-5836364d8bd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:17:38.197751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ed3c8-9ab1-4717-a5e9-5836364d8bd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:38.198221 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ed3c8-9ab1-4717-a5e9-5836364d8bd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:38.198531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ed3c8-9ab1-4717-a5e9-5836364d8bd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:38.200458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-fb3ed3c8-9ab1-4717-a5e9-5836364d8bd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:38.306970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ed3c8-9ab1-4717-a5e9-5836364d8bd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:17:38.308376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ed3c8-9ab1-4717-a5e9-5836364d8bd2 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:17:38.327099 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 750/1487] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:17:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:17:40.623233 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01247c4-1bf0-4d77-a61c-11efcb6316fe None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:40.623865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01247c4-1bf0-4d77-a61c-11efcb6316fe None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:40.624002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01247c4-1bf0-4d77-a61c-11efcb6316fe None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:40.625279 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 738/1488] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:17:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:17:40.630521 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f247bf-1103-48dc-b6ad-30baea34078f None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:40.630958 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f247bf-1103-48dc-b6ad-30baea34078f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:40.631053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f247bf-1103-48dc-b6ad-30baea34078f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:40.697705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-80f247bf-1103-48dc-b6ad-30baea34078f None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:17:40.698888 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-80f247bf-1103-48dc-b6ad-30baea34078f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:40.751638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:40.796594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 751/1489] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:17:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:17:41.098189 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01247c4-1bf0-4d77-a61c-11efcb6316fe None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:17:41.099830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b01247c4-1bf0-4d77-a61c-11efcb6316fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:41.137391 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:41.210494 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01247c4-1bf0-4d77-a61c-11efcb6316fe None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:17:41.283118 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829bb8c-0152-47fb-b279-8757e887f9fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:17:41.283720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829bb8c-0152-47fb-b279-8757e887f9fd service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:41.283996 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829bb8c-0152-47fb-b279-8757e887f9fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:41.284264 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829bb8c-0152-47fb-b279-8757e887f9fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:41.285880 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e829bb8c-0152-47fb-b279-8757e887f9fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:41.365840 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:41.394711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829bb8c-0152-47fb-b279-8757e887f9fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:17:41.396125 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829bb8c-0152-47fb-b279-8757e887f9fd service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:17:41.414805 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 739/1490] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:17:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:17:43.790795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48309e59-6114-4efb-8194-42e1d2ecf114 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:43.791372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48309e59-6114-4efb-8194-42e1d2ecf114 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:43.791372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48309e59-6114-4efb-8194-42e1d2ecf114 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:43.793072 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 752/1491] 10.222.0.60 () {54 vars in 927 bytes} [Wed Sep 28 12:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:17:43.799294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644990e-4909-4f93-9d5d-8eeabbcd19eb None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:43.799747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644990e-4909-4f93-9d5d-8eeabbcd19eb None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:43.799858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644990e-4909-4f93-9d5d-8eeabbcd19eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:43.867221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-6644990e-4909-4f93-9d5d-8eeabbcd19eb None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:17:43.868469 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-6644990e-4909-4f93-9d5d-8eeabbcd19eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:43.965975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 740/1492] 10.222.0.60 () {58 vars in 1024 bytes} [Wed Sep 28 12:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:17:44.256876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48309e59-6114-4efb-8194-42e1d2ecf114 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:17:44.258425 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-48309e59-6114-4efb-8194-42e1d2ecf114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:44.362163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48309e59-6114-4efb-8194-42e1d2ecf114 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:17:44.433919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f886b-fcd6-4bee-ad78-46fce431cc7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:17:44.434523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f886b-fcd6-4bee-ad78-46fce431cc7a service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:17:44.434802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f886b-fcd6-4bee-ad78-46fce431cc7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:17:44.435637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f886b-fcd6-4bee-ad78-46fce431cc7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:17:44.437266 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-8c2f886b-fcd6-4bee-ad78-46fce431cc7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:17:44.445716 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:44.543716 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f886b-fcd6-4bee-ad78-46fce431cc7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:17:44.545747 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f886b-fcd6-4bee-ad78-46fce431cc7a service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:17:44.563813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 753/1493] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:17:53.964470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=82 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:17:54.561556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=50 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:19:12.734845 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909e805-a614-4b81-a3b9-984ea6038136 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:19:12.735445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909e805-a614-4b81-a3b9-984ea6038136 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:19:12.735445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909e805-a614-4b81-a3b9-984ea6038136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:19:12.811330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-7909e805-a614-4b81-a3b9-984ea6038136 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:19:12.812536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7909e805-a614-4b81-a3b9-984ea6038136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:19:12.908372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 741/1494] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:19:12.916517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ed4b1-6322-4cd3-be18-a8600f6cb8d4 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:19:12.917018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ed4b1-6322-4cd3-be18-a8600f6cb8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:19:12.917122 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ed4b1-6322-4cd3-be18-a8600f6cb8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:19:12.993286 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-c35ed4b1-6322-4cd3-be18-a8600f6cb8d4 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:19:12.994402 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-c35ed4b1-6322-4cd3-be18-a8600f6cb8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:19:13.089023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 754/1495] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:19:13.093897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909e805-a614-4b81-a3b9-984ea6038136 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:19:13.095102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7909e805-a614-4b81-a3b9-984ea6038136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:19:13.107070 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909e805-a614-4b81-a3b9-984ea6038136 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:19:13.169555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:19:13.205004 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2082b5-a72d-43dc-94b3-416589f9ad15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:19:13.205653 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2082b5-a72d-43dc-94b3-416589f9ad15 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:19:13.205932 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2082b5-a72d-43dc-94b3-416589f9ad15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:19:13.206200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2082b5-a72d-43dc-94b3-416589f9ad15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:19:13.206847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2082b5-a72d-43dc-94b3-416589f9ad15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:19:13.207150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2082b5-a72d-43dc-94b3-416589f9ad15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:19:13.208167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: WARNING py.warnings [None req-ed2082b5-a72d-43dc-94b3-416589f9ad15 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 12:19:13.208167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: warnings.warn(msg) Sep 28 12:19:13.208167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: Sep 28 12:19:13.209052 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2082b5-a72d-43dc-94b3-416589f9ad15 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:19:13.221169 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 742/1496] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:19:13.229964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ed4b1-6322-4cd3-be18-a8600f6cb8d4 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:19:13.238932 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ed4b1-6322-4cd3-be18-a8600f6cb8d4 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:19:13.299235 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:19:13.332074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef29f2-2e73-4720-b609-586b5f622e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:19:13.332668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef29f2-2e73-4720-b609-586b5f622e07 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:19:13.332941 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef29f2-2e73-4720-b609-586b5f622e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:19:13.333239 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef29f2-2e73-4720-b609-586b5f622e07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:19:13.333859 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef29f2-2e73-4720-b609-586b5f622e07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:19:13.337153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: WARNING py.warnings [None req-31ef29f2-2e73-4720-b609-586b5f622e07 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 12:19:13.337153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: warnings.warn(msg) Sep 28 12:19:13.337153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: Sep 28 12:19:13.338330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef29f2-2e73-4720-b609-586b5f622e07 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:19:13.395657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 755/1497] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:19:13 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:19:13.400064 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2082b5-a72d-43dc-94b3-416589f9ad15 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:19:13.401647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-ed2082b5-a72d-43dc-94b3-416589f9ad15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:19:13.436173 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2082b5-a72d-43dc-94b3-416589f9ad15 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:19:13.535931 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293e1d6-55cf-4151-b0d3-660b10768d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:19:13.536538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293e1d6-55cf-4151-b0d3-660b10768d35 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:19:13.536824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293e1d6-55cf-4151-b0d3-660b10768d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:19:13.537099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293e1d6-55cf-4151-b0d3-660b10768d35 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:19:13.537760 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293e1d6-55cf-4151-b0d3-660b10768d35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:19:13.538772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: WARNING py.warnings [None req-8293e1d6-55cf-4151-b0d3-660b10768d35 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 12:19:13.538772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: warnings.warn(msg) Sep 28 12:19:13.538772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: Sep 28 12:19:13.540083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293e1d6-55cf-4151-b0d3-660b10768d35 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:19:13.582854 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 743/1498] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:19:13 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:19:13.587131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef29f2-2e73-4720-b609-586b5f622e07 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:19:13.588624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-31ef29f2-2e73-4720-b609-586b5f622e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:19:13.649062 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef29f2-2e73-4720-b609-586b5f622e07 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:19:13.729877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:19:13.740213 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cf82b-9b18-432c-a814-5b365e2201b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:19:13.740810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cf82b-9b18-432c-a814-5b365e2201b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:19:13.741086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cf82b-9b18-432c-a814-5b365e2201b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:19:13.741376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cf82b-9b18-432c-a814-5b365e2201b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:19:13.742001 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562cf82b-9b18-432c-a814-5b365e2201b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:19:13.743664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: WARNING py.warnings [None req-562cf82b-9b18-432c-a814-5b365e2201b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 12:19:13.743664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: warnings.warn(msg) Sep 28 12:19:13.743664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: Sep 28 12:19:13.744536 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562cf82b-9b18-432c-a814-5b365e2201b6 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:19:13.757407 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 756/1499] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:19:13.762397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293e1d6-55cf-4151-b0d3-660b10768d35 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:19:13.764093 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8293e1d6-55cf-4151-b0d3-660b10768d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:19:13.775545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293e1d6-55cf-4151-b0d3-660b10768d35 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:19:13.800797 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:19:13.873640 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed121a-1ae6-4ae9-b7db-68954be83c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:19:13.874154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed121a-1ae6-4ae9-b7db-68954be83c2d admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:19:13.874433 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed121a-1ae6-4ae9-b7db-68954be83c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:19:13.874703 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed121a-1ae6-4ae9-b7db-68954be83c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:19:13.875341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed121a-1ae6-4ae9-b7db-68954be83c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:19:13.876361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: WARNING py.warnings [None req-23ed121a-1ae6-4ae9-b7db-68954be83c2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 12:19:13.876361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: warnings.warn(msg) Sep 28 12:19:13.876361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: Sep 28 12:19:13.877263 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed121a-1ae6-4ae9-b7db-68954be83c2d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:19:13.889494 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 744/1500] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:19:13.893762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cf82b-9b18-432c-a814-5b365e2201b6 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:19:13.895349 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-562cf82b-9b18-432c-a814-5b365e2201b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:19:13.928650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cf82b-9b18-432c-a814-5b365e2201b6 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:19:14.021479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0409a-9684-46e6-8b51-4b710dd283e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:19:14.022129 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0409a-9684-46e6-8b51-4b710dd283e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:19:14.022402 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0409a-9684-46e6-8b51-4b710dd283e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:19:14.022672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0409a-9684-46e6-8b51-4b710dd283e9 admin admin] PATH_INFO: `/v3/projects/f8e63228e5f945ea8ab79e3be7787f70/users/a9d543ef99804d19bc1b45d32553f4da/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:19:14.044773 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0409a-9684-46e6-8b51-4b710dd283e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8e63228e5f945ea8ab79e3be7787f70, user_id=a9d543ef99804d19bc1b45d32553f4da, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:19:14.045808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: WARNING py.warnings [None req-60c0409a-9684-46e6-8b51-4b710dd283e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 12:19:14.045808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: warnings.warn(msg) Sep 28 12:19:14.045808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: Sep 28 12:19:14.047689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0409a-9684-46e6-8b51-4b710dd283e9 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:19:14.071538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 757/1501] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:19:13 2022] PUT /identity/v3/projects/f8e63228e5f945ea8ab79e3be7787f70/users/a9d543ef99804d19bc1b45d32553f4da/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:19:14.083575 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6ed1b-0cbf-4b0d-a6f1-dc3410bde435 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:19:14.083869 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6ed1b-0cbf-4b0d-a6f1-dc3410bde435 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:19:14.084178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6ed1b-0cbf-4b0d-a6f1-dc3410bde435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:19:14.141843 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-d7e6ed1b-0cbf-4b0d-a6f1-dc3410bde435 None None] MFA Rules not processed for user `a9d543ef99804d19bc1b45d32553f4da`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:19:14.142964 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d7e6ed1b-0cbf-4b0d-a6f1-dc3410bde435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:19:14.224264 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 745/1502] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:19:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:19:14.238471 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78445491-3fdc-4d0c-87b2-cd1196503d91 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:19:14.238768 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78445491-3fdc-4d0c-87b2-cd1196503d91 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:19:14.239034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78445491-3fdc-4d0c-87b2-cd1196503d91 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:19:14.240388 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 758/1503] 10.222.0.60 () {54 vars in 963 bytes} [Wed Sep 28 12:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:19:14.245378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bfd80-24e1-4c13-923a-b451fc7edef5 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:19:14.245671 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bfd80-24e1-4c13-923a-b451fc7edef5 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:19:14.245952 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bfd80-24e1-4c13-923a-b451fc7edef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:19:14.277784 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:19:14.440453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-970bfd80-24e1-4c13-923a-b451fc7edef5 None None] MFA Rules not processed for user `2354a4fd72cc42c58d6830eecb1e6253`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:19:14.441717 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-970bfd80-24e1-4c13-923a-b451fc7edef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:19:14.519509 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 746/1504] 10.222.0.60 () {58 vars in 1060 bytes} [Wed Sep 28 12:19:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:19:14.528088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78445491-3fdc-4d0c-87b2-cd1196503d91 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:19:14.530135 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-78445491-3fdc-4d0c-87b2-cd1196503d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:19:14.580277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:19:14.635110 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78445491-3fdc-4d0c-87b2-cd1196503d91 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:19:14.706721 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f5ca3-b475-40c2-b080-29f51d00ddd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:19:14.707252 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f5ca3-b475-40c2-b080-29f51d00ddd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:19:14.707527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f5ca3-b475-40c2-b080-29f51d00ddd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:19:14.707793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f5ca3-b475-40c2-b080-29f51d00ddd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:19:14.709378 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-280f5ca3-b475-40c2-b080-29f51d00ddd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:19:14.807106 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f5ca3-b475-40c2-b080-29f51d00ddd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:19:14.808527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f5ca3-b475-40c2-b080-29f51d00ddd7 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:19:14.811964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:19:14.827054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 759/1505] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:19:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:19:14.873368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c79a0c-29ba-4707-b77f-917a5a04f843 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:19:14.873817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c79a0c-29ba-4707-b77f-917a5a04f843 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:19:14.873929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c79a0c-29ba-4707-b77f-917a5a04f843 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:19:14.875184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 747/1506] 10.222.0.60 () {52 vars in 830 bytes} [Wed Sep 28 12:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:19:14.878737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f5ca3-b475-40c2-b080-29f51d00ddd7 service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:19:14.879948 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-280f5ca3-b475-40c2-b080-29f51d00ddd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:19:14.977334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f5ca3-b475-40c2-b080-29f51d00ddd7 service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:19:15.068557 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb7f02-9ff8-4174-9f9c-73ecbaba7b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:19:15.069204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb7f02-9ff8-4174-9f9c-73ecbaba7b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:19:15.069485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb7f02-9ff8-4174-9f9c-73ecbaba7b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:19:15.069754 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fb7f02-9ff8-4174-9f9c-73ecbaba7b5c admin admin] PATH_INFO: `/v3/users/a9d543ef99804d19bc1b45d32553f4da` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:19:15.082300 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb7f02-9ff8-4174-9f9c-73ecbaba7b5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9d543ef99804d19bc1b45d32553f4da)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:19:15.083318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: WARNING py.warnings [None req-66fb7f02-9ff8-4174-9f9c-73ecbaba7b5c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 12:19:15.083318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: warnings.warn(msg) Sep 28 12:19:15.083318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: Sep 28 12:19:15.084506 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fb7f02-9ff8-4174-9f9c-73ecbaba7b5c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:19:15.094515 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:19:15.152071 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-66fb7f02-9ff8-4174-9f9c-73ecbaba7b5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9d543ef99804d19bc1b45d32553f4da'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:19:15.152499 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-66fb7f02-9ff8-4174-9f9c-73ecbaba7b5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9d543ef99804d19bc1b45d32553f4da'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:19:15.160947 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-66fb7f02-9ff8-4174-9f9c-73ecbaba7b5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9d543ef99804d19bc1b45d32553f4da'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:19:15.161539 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-66fb7f02-9ff8-4174-9f9c-73ecbaba7b5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9d543ef99804d19bc1b45d32553f4da'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:19:15.178362 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-66fb7f02-9ff8-4174-9f9c-73ecbaba7b5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9d543ef99804d19bc1b45d32553f4da'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:19:15.206123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 760/1507] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:19:14 2022] DELETE /identity/v3/users/a9d543ef99804d19bc1b45d32553f4da => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:19:15.210592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0f6ef-3b16-4e2d-a55a-95e319a4271e None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:19:15.210875 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0f6ef-3b16-4e2d-a55a-95e319a4271e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:19:15.211147 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0f6ef-3b16-4e2d-a55a-95e319a4271e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:19:15.270637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-2dd0f6ef-3b16-4e2d-a55a-95e319a4271e None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:19:15.271760 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2dd0f6ef-3b16-4e2d-a55a-95e319a4271e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:19:15.345257 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:19:15.358200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 748/1508] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:19:15.367773 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb7f02-9ff8-4174-9f9c-73ecbaba7b5c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:19:15.368990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-66fb7f02-9ff8-4174-9f9c-73ecbaba7b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:19:15.471412 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fb7f02-9ff8-4174-9f9c-73ecbaba7b5c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:19:15.528962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:19:15.548985 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ee8b3-e884-4f0b-85b5-68d63818a68e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:19:15.549631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ee8b3-e884-4f0b-85b5-68d63818a68e service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:19:15.549911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ee8b3-e884-4f0b-85b5-68d63818a68e service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:19:15.550185 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ee8b3-e884-4f0b-85b5-68d63818a68e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:19:15.551763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-8c6ee8b3-e884-4f0b-85b5-68d63818a68e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:19:15.658605 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ee8b3-e884-4f0b-85b5-68d63818a68e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:19:15.660017 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ee8b3-e884-4f0b-85b5-68d63818a68e service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:19:15.678616 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 761/1509] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:19:16.026583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0f6ef-3b16-4e2d-a55a-95e319a4271e None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:19:16.027837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2dd0f6ef-3b16-4e2d-a55a-95e319a4271e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:19:16.039074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0f6ef-3b16-4e2d-a55a-95e319a4271e None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:19:16.136725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926562c-59c3-419c-a074-854a8db999f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:19:16.137389 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926562c-59c3-419c-a074-854a8db999f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:19:16.137675 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926562c-59c3-419c-a074-854a8db999f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:19:16.137944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926562c-59c3-419c-a074-854a8db999f7 admin admin] PATH_INFO: `/v3/projects/f8e63228e5f945ea8ab79e3be7787f70` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:19:16.148177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926562c-59c3-419c-a074-854a8db999f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8e63228e5f945ea8ab79e3be7787f70)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:19:16.149916 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: WARNING py.warnings [None req-1926562c-59c3-419c-a074-854a8db999f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 12:19:16.149916 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: warnings.warn(msg) Sep 28 12:19:16.149916 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: Sep 28 12:19:16.151082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926562c-59c3-419c-a074-854a8db999f7 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:19:16.207857 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1926562c-59c3-419c-a074-854a8db999f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8e63228e5f945ea8ab79e3be7787f70', 'request_id': 'req-1926562c-59c3-419c-a074-854a8db999f7'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:19:16.223072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1926562c-59c3-419c-a074-854a8db999f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8e63228e5f945ea8ab79e3be7787f70', 'request_id': 'req-1926562c-59c3-419c-a074-854a8db999f7'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:19:16.232702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1926562c-59c3-419c-a074-854a8db999f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8e63228e5f945ea8ab79e3be7787f70', 'request_id': 'req-1926562c-59c3-419c-a074-854a8db999f7'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:19:16.254880 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1926562c-59c3-419c-a074-854a8db999f7 admin admin] The token cache is being invalidate because project f8e63228e5f945ea8ab79e3be7787f70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:19:16.255198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1926562c-59c3-419c-a074-854a8db999f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:19:16.255629 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1926562c-59c3-419c-a074-854a8db999f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:19:16.256993 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 749/1510] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:19:16 2022] DELETE /identity/v3/projects/f8e63228e5f945ea8ab79e3be7787f70 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:19:25.676379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=62 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:19:26.252889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:20:42.805875 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f3975-9cbb-4bc0-9095-7d0b25768671 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:20:42.806499 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f3975-9cbb-4bc0-9095-7d0b25768671 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:20:42.806997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f3975-9cbb-4bc0-9095-7d0b25768671 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:20:42.808726 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 12:20:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:20:42.814457 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31976e53-c436-424c-9b2d-10715fbbd589 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:20:42.814935 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31976e53-c436-424c-9b2d-10715fbbd589 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:20:42.815039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31976e53-c436-424c-9b2d-10715fbbd589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:20:42.903442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-31976e53-c436-424c-9b2d-10715fbbd589 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:20:42.904669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-31976e53-c436-424c-9b2d-10715fbbd589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:20:43.015642 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 12:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:20:43.183821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f3975-9cbb-4bc0-9095-7d0b25768671 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:20:43.186058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-471f3975-9cbb-4bc0-9095-7d0b25768671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:20:43.333657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f3975-9cbb-4bc0-9095-7d0b25768671 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:20:43.340203 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3b454-b553-4b97-ad55-ae640fc6f728 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:20:43.340812 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3b454-b553-4b97-ad55-ae640fc6f728 service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:20:43.341095 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3b454-b553-4b97-ad55-ae640fc6f728 service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:20:43.341398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3b454-b553-4b97-ad55-ae640fc6f728 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:20:43.352865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3b454-b553-4b97-ad55-ae640fc6f728 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:20:43.355418 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3b454-b553-4b97-ad55-ae640fc6f728 service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:20:43.369683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 763/1513] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:20:44.463325 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f556ae-ec9b-4459-a33d-19d8cc21c697 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:20:44.463859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f556ae-ec9b-4459-a33d-19d8cc21c697 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:20:44.463859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f556ae-ec9b-4459-a33d-19d8cc21c697 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:20:44.465290 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 12:20:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:20:44.471557 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92d53e-34ce-44f5-b297-cfa85322eebd None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:20:44.472039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92d53e-34ce-44f5-b297-cfa85322eebd None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:20:44.472182 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92d53e-34ce-44f5-b297-cfa85322eebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:20:44.492206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-dc92d53e-34ce-44f5-b297-cfa85322eebd None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:20:44.493448 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-dc92d53e-34ce-44f5-b297-cfa85322eebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:20:44.501152 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 12:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:20:44.654205 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f556ae-ec9b-4459-a33d-19d8cc21c697 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:20:44.669123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f556ae-ec9b-4459-a33d-19d8cc21c697 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:20:44.674518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b61b7-a350-4fc6-9078-a50cd4894e81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:20:44.675121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b61b7-a350-4fc6-9078-a50cd4894e81 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:20:44.675404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b61b7-a350-4fc6-9078-a50cd4894e81 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:20:44.675678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b61b7-a350-4fc6-9078-a50cd4894e81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:20:44.683858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b61b7-a350-4fc6-9078-a50cd4894e81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:20:44.685469 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2b61b7-a350-4fc6-9078-a50cd4894e81 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:20:44.694700 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 752/1516] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:20:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:20:44.915109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93343d-187a-4585-bf99-7dbbc20b9e6f None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:20:44.915388 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93343d-187a-4585-bf99-7dbbc20b9e6f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:20:44.915663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93343d-187a-4585-bf99-7dbbc20b9e6f None None] PATH_INFO: `/v3` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:20:44.916937 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 765/1517] 10.222.0.52 () {54 vars in 925 bytes} [Wed Sep 28 12:20:44 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:20:44.923428 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74d8b9-3223-4cf7-851e-08c98c1d62b6 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:20:44.923717 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74d8b9-3223-4cf7-851e-08c98c1d62b6 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:20:44.923992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74d8b9-3223-4cf7-851e-08c98c1d62b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:20:44.983511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-cb74d8b9-3223-4cf7-851e-08c98c1d62b6 None None] MFA Rules not processed for user `9a2322b1a76d40e49545f5677e247489`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:20:44.984630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-cb74d8b9-3223-4cf7-851e-08c98c1d62b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:20:45.001047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:20:45.055735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 753/1518] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Sep 28 12:20:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:20:54.487259 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=56 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:20:55.020195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560bb92-2fa5-4671-a822-bc0ef3852af6 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:20:55.020478 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560bb92-2fa5-4671-a822-bc0ef3852af6 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:20:55.020779 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a560bb92-2fa5-4671-a822-bc0ef3852af6 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:20:55.022241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 766/1519] 10.222.0.60 () {54 vars in 961 bytes} [Wed Sep 28 12:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:20:55.027518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85e9ca-29df-4796-9041-c4ac01f32b4c None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:20:55.028022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85e9ca-29df-4796-9041-c4ac01f32b4c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:20:55.028126 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85e9ca-29df-4796-9041-c4ac01f32b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:20:55.048108 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-9d85e9ca-29df-4796-9041-c4ac01f32b4c None None] MFA Rules not processed for user `9a2322b1a76d40e49545f5677e247489`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:20:55.049262 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-9d85e9ca-29df-4796-9041-c4ac01f32b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:20:55.058639 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 754/1520] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Sep 28 12:20:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:20:55.066358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560bb92-2fa5-4671-a822-bc0ef3852af6 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:20:55.080407 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a560bb92-2fa5-4671-a822-bc0ef3852af6 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:20:55.085347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bce7a2-aa89-4e75-89d4-6f8c60271c2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:20:55.085938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bce7a2-aa89-4e75-89d4-6f8c60271c2e service placement] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:20:55.086218 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bce7a2-aa89-4e75-89d4-6f8c60271c2e service placement] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:20:55.086543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bce7a2-aa89-4e75-89d4-6f8c60271c2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:20:55.094812 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bce7a2-aa89-4e75-89d4-6f8c60271c2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:20:55.096383 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bce7a2-aa89-4e75-89d4-6f8c60271c2e service placement] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:20:55.106523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 767/1521] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:20:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:20:56.196945 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841f305-3a85-4695-8794-9bb5256ff4aa None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:20:56.197472 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841f305-3a85-4695-8794-9bb5256ff4aa None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:20:56.197575 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841f305-3a85-4695-8794-9bb5256ff4aa None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:20:56.198864 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 12:20:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:20:56.205440 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc88296-98b1-4e9a-818f-93c82f5e6eaf None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:20:56.205881 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc88296-98b1-4e9a-818f-93c82f5e6eaf None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:20:56.205983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc88296-98b1-4e9a-818f-93c82f5e6eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:20:56.226210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-efc88296-98b1-4e9a-818f-93c82f5e6eaf None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:20:56.227372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-efc88296-98b1-4e9a-818f-93c82f5e6eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:20:56.235268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 12:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:20:56.392064 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841f305-3a85-4695-8794-9bb5256ff4aa None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:20:56.407101 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841f305-3a85-4695-8794-9bb5256ff4aa None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:20:56.412420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a7b74-d1b9-421d-8698-cc622f22d8e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:20:56.413018 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a7b74-d1b9-421d-8698-cc622f22d8e2 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:20:56.413329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a7b74-d1b9-421d-8698-cc622f22d8e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:20:56.413603 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a7b74-d1b9-421d-8698-cc622f22d8e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:20:56.421780 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a7b74-d1b9-421d-8698-cc622f22d8e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:20:56.423341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a7b74-d1b9-421d-8698-cc622f22d8e2 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:20:56.433395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 756/1524] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:20:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:20:57.505256 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75170a47-bb82-4201-b293-e4ba70b657b1 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:20:57.507245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75170a47-bb82-4201-b293-e4ba70b657b1 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:20:57.507245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75170a47-bb82-4201-b293-e4ba70b657b1 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:20:57.507245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 12:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:20:57.514161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b679af1-9ff1-4c50-9e5b-664b97f9415a None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:20:57.514650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b679af1-9ff1-4c50-9e5b-664b97f9415a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:20:57.514751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b679af1-9ff1-4c50-9e5b-664b97f9415a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:20:57.533667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-8b679af1-9ff1-4c50-9e5b-664b97f9415a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:20:57.534848 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8b679af1-9ff1-4c50-9e5b-664b97f9415a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:20:57.543221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 12:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:20:57.696390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75170a47-bb82-4201-b293-e4ba70b657b1 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:20:57.710499 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75170a47-bb82-4201-b293-e4ba70b657b1 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:20:57.715900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1ee4e-ee03-44dc-b29e-0fe89e2d11ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:20:57.716483 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1ee4e-ee03-44dc-b29e-0fe89e2d11ee service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:20:57.716757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1ee4e-ee03-44dc-b29e-0fe89e2d11ee service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:20:57.717022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1ee4e-ee03-44dc-b29e-0fe89e2d11ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:20:57.725083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1ee4e-ee03-44dc-b29e-0fe89e2d11ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:20:57.727396 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1ee4e-ee03-44dc-b29e-0fe89e2d11ee service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:20:57.736493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 770/1527] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:20:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:21:01.788416 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36261e07-daa3-40f7-80ce-675aa82b9cfb None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:21:01.789002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36261e07-daa3-40f7-80ce-675aa82b9cfb None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:21:01.789002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36261e07-daa3-40f7-80ce-675aa82b9cfb None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:21:01.790990 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 12:21:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:21:01.796873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73579a1-ef23-4149-83b3-72a760d87d93 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:21:01.797394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73579a1-ef23-4149-83b3-72a760d87d93 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:21:01.797571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73579a1-ef23-4149-83b3-72a760d87d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:21:01.818528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-b73579a1-ef23-4149-83b3-72a760d87d93 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:21:01.819877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b73579a1-ef23-4149-83b3-72a760d87d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:21:01.827814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 12:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:21:02.119093 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36261e07-daa3-40f7-80ce-675aa82b9cfb None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:21:02.120823 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-36261e07-daa3-40f7-80ce-675aa82b9cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:21:02.210488 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36261e07-daa3-40f7-80ce-675aa82b9cfb None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:21:02.215129 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0225b-02d9-435e-948d-8daee8583ba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:21:02.215743 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0225b-02d9-435e-948d-8daee8583ba1 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:21:02.216033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0225b-02d9-435e-948d-8daee8583ba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:21:02.216317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0225b-02d9-435e-948d-8daee8583ba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:21:02.225312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0225b-02d9-435e-948d-8daee8583ba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:21:02.227039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0225b-02d9-435e-948d-8daee8583ba1 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:21:02.236455 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 759/1530] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:21:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:21:04.399411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b316b09-fe7b-4cbd-9c9d-0bb86d3466f8 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:21:04.399959 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b316b09-fe7b-4cbd-9c9d-0bb86d3466f8 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:21:04.399959 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b316b09-fe7b-4cbd-9c9d-0bb86d3466f8 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:21:04.401224 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 12:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:21:04.406570 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b5640-c387-4ff8-922e-b783d55d19ea None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:21:04.407006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b5640-c387-4ff8-922e-b783d55d19ea None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:21:04.407112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b5640-c387-4ff8-922e-b783d55d19ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:21:04.426993 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-d23b5640-c387-4ff8-922e-b783d55d19ea None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:21:04.428177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d23b5640-c387-4ff8-922e-b783d55d19ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:21:04.436740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 12:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:21:04.724724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b316b09-fe7b-4cbd-9c9d-0bb86d3466f8 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:21:04.736542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b316b09-fe7b-4cbd-9c9d-0bb86d3466f8 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:21:04.740342 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0e5a6-a4ab-401d-a7ef-3887954ade02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:21:04.740926 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0e5a6-a4ab-401d-a7ef-3887954ade02 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:21:04.741235 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0e5a6-a4ab-401d-a7ef-3887954ade02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:21:04.741507 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0e5a6-a4ab-401d-a7ef-3887954ade02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:21:04.750102 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0e5a6-a4ab-401d-a7ef-3887954ade02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:21:04.751505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0e5a6-a4ab-401d-a7ef-3887954ade02 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:21:04.760525 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 773/1533] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:21:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:21:08.339084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf01c51-b4e3-480c-95bf-805509b6910d None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:21:08.339612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf01c51-b4e3-480c-95bf-805509b6910d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:21:08.339612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf01c51-b4e3-480c-95bf-805509b6910d None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:21:08.340892 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 12:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:21:08.346222 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faa412-1c94-42cc-9b4f-f75ca5c1bc96 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:21:08.346676 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faa412-1c94-42cc-9b4f-f75ca5c1bc96 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:21:08.346781 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13faa412-1c94-42cc-9b4f-f75ca5c1bc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:21:08.366028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-13faa412-1c94-42cc-9b4f-f75ca5c1bc96 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:21:08.367310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-13faa412-1c94-42cc-9b4f-f75ca5c1bc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:21:08.375381 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 12:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:21:08.678156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf01c51-b4e3-480c-95bf-805509b6910d None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:21:08.680421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5cf01c51-b4e3-480c-95bf-805509b6910d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:21:08.744818 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:21:08.759549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf01c51-b4e3-480c-95bf-805509b6910d None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:21:08.763097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179791ad-58bf-47b9-87d4-74d8891024ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:21:08.763698 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179791ad-58bf-47b9-87d4-74d8891024ba service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:21:08.764095 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179791ad-58bf-47b9-87d4-74d8891024ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:21:08.764376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179791ad-58bf-47b9-87d4-74d8891024ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:21:08.773258 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179791ad-58bf-47b9-87d4-74d8891024ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:21:08.774702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179791ad-58bf-47b9-87d4-74d8891024ba service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:21:08.783961 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 762/1536] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:21:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:21:10.922524 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43287b0a-4afe-426b-9d1c-7a7081e0972b None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:21:10.923060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43287b0a-4afe-426b-9d1c-7a7081e0972b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:21:10.923060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43287b0a-4afe-426b-9d1c-7a7081e0972b None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:21:10.924309 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 12:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:21:10.930248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce953e7-c2c4-427d-8cf0-149b489ec38d None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:21:10.930681 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce953e7-c2c4-427d-8cf0-149b489ec38d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:21:10.930776 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce953e7-c2c4-427d-8cf0-149b489ec38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:21:10.950661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-2ce953e7-c2c4-427d-8cf0-149b489ec38d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:21:10.951849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2ce953e7-c2c4-427d-8cf0-149b489ec38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:21:10.959644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 12:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:21:11.263425 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43287b0a-4afe-426b-9d1c-7a7081e0972b None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:21:11.265248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-43287b0a-4afe-426b-9d1c-7a7081e0972b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:21:11.278142 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43287b0a-4afe-426b-9d1c-7a7081e0972b None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:21:11.282051 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f8562-8c38-4f4c-b997-b5968ae3e45b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:21:11.282710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f8562-8c38-4f4c-b997-b5968ae3e45b service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:21:11.283030 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f8562-8c38-4f4c-b997-b5968ae3e45b service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:21:11.283354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f8562-8c38-4f4c-b997-b5968ae3e45b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:21:11.293417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f8562-8c38-4f4c-b997-b5968ae3e45b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:21:11.294824 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f8562-8c38-4f4c-b997-b5968ae3e45b service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:21:11.303962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 776/1539] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:21:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:21:20.945072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=11 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:21:21.299858 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=33 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:35.746393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196d8a4-140d-4aee-a51a-b3f284daeb38 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:35.747589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196d8a4-140d-4aee-a51a-b3f284daeb38 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:35.748088 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196d8a4-140d-4aee-a51a-b3f284daeb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:35.775831 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-8196d8a4-140d-4aee-a51a-b3f284daeb38 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:35.777603 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8196d8a4-140d-4aee-a51a-b3f284daeb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:35.790364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 764/1540] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:22:35.801269 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21247c3-2125-4fbc-b693-cf517565c897 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:35.802546 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21247c3-2125-4fbc-b693-cf517565c897 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:35.803702 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21247c3-2125-4fbc-b693-cf517565c897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:35.831998 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-d21247c3-2125-4fbc-b693-cf517565c897 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:35.833859 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d21247c3-2125-4fbc-b693-cf517565c897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:35.848704 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 777/1541] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:35.855092 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196d8a4-140d-4aee-a51a-b3f284daeb38 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:35.867898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196d8a4-140d-4aee-a51a-b3f284daeb38 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:35.876642 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b908994-2769-488e-9dcf-7c6960863adb None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:35.876978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b908994-2769-488e-9dcf-7c6960863adb None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:35.877237 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b908994-2769-488e-9dcf-7c6960863adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:35.884529 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03944902-46df-450d-97cc-65ed894979c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:35.887954 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03944902-46df-450d-97cc-65ed894979c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:35.888311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03944902-46df-450d-97cc-65ed894979c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:35.888588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03944902-46df-450d-97cc-65ed894979c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:35.891013 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03944902-46df-450d-97cc-65ed894979c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:22:35.891332 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03944902-46df-450d-97cc-65ed894979c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:35.898171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03944902-46df-450d-97cc-65ed894979c4 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:35.919198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7b908994-2769-488e-9dcf-7c6960863adb None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:35.926258 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7b908994-2769-488e-9dcf-7c6960863adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:35.931620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 765/1542] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:22:35.944233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 778/1543] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:22:35.951021 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03944902-46df-450d-97cc-65ed894979c4 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:35.959084 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79a225-220a-4d32-b813-6b14fedb0d3b None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:35.959558 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79a225-220a-4d32-b813-6b14fedb0d3b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:35.959944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79a225-220a-4d32-b813-6b14fedb0d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:35.969946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03944902-46df-450d-97cc-65ed894979c4 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:35.979224 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7165326-2988-468c-bb34-fd5f70750a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:35.981282 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165326-2988-468c-bb34-fd5f70750a7a admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:35.981566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165326-2988-468c-bb34-fd5f70750a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:35.981842 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165326-2988-468c-bb34-fd5f70750a7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:35.984218 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7165326-2988-468c-bb34-fd5f70750a7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:35.985378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: WARNING py.warnings [None req-f7165326-2988-468c-bb34-fd5f70750a7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 12:22:35.985378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: warnings.warn(msg) Sep 28 12:22:35.985378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: Sep 28 12:22:35.986543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7165326-2988-468c-bb34-fd5f70750a7a admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.020725 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-1c79a225-220a-4d32-b813-6b14fedb0d3b None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:36.022131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-1c79a225-220a-4d32-b813-6b14fedb0d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:36.048380 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 779/1544] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:36.056605 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef607f-e6a5-4743-b4ab-65966af09fe2 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.056908 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef607f-e6a5-4743-b4ab-65966af09fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.057159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef607f-e6a5-4743-b4ab-65966af09fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.092376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 766/1545] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:35 2022] POST /identity/v3/projects => generated 389 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:36.096824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7165326-2988-468c-bb34-fd5f70750a7a admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.111652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-27ef607f-e6a5-4743-b4ab-65966af09fe2 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:36.112922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-27ef607f-e6a5-4743-b4ab-65966af09fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:36.203888 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7165326-2988-468c-bb34-fd5f70750a7a admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.205470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 780/1546] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:36.210090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef607f-e6a5-4743-b4ab-65966af09fe2 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.213061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b5973-79c9-466a-971f-598e93a484e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.213791 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b5973-79c9-466a-971f-598e93a484e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.214085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b5973-79c9-466a-971f-598e93a484e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.214364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b5973-79c9-466a-971f-598e93a484e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.215023 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b5973-79c9-466a-971f-598e93a484e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:22:36.215331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b5973-79c9-466a-971f-598e93a484e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.216949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b5973-79c9-466a-971f-598e93a484e2 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.222013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef607f-e6a5-4743-b4ab-65966af09fe2 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.229210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e9607-4f0b-4f05-a7bf-e301d76337fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.230010 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e9607-4f0b-4f05-a7bf-e301d76337fb admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.230211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 767/1547] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:22:36.231471 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c12c27-de3d-49d1-9602-ddddbea0c098 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.231761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c12c27-de3d-49d1-9602-ddddbea0c098 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.232039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c12c27-de3d-49d1-9602-ddddbea0c098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.237838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e9607-4f0b-4f05-a7bf-e301d76337fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.238131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e9607-4f0b-4f05-a7bf-e301d76337fb admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.238951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e9607-4f0b-4f05-a7bf-e301d76337fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.239991 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: WARNING py.warnings [None req-db1e9607-4f0b-4f05-a7bf-e301d76337fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 12:22:36.239991 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: warnings.warn(msg) Sep 28 12:22:36.239991 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: Sep 28 12:22:36.241755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e9607-4f0b-4f05-a7bf-e301d76337fb admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.262123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-67c12c27-de3d-49d1-9602-ddddbea0c098 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:36.263341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-67c12c27-de3d-49d1-9602-ddddbea0c098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:36.266369 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 781/1548] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/users => generated 295 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:36.268931 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e20583-cee2-4c5e-85c2-3ea84f95ba4f None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.268931 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e20583-cee2-4c5e-85c2-3ea84f95ba4f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.268931 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e20583-cee2-4c5e-85c2-3ea84f95ba4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.274355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 768/1549] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:36.275555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055abc2-a42d-4996-b9fe-0205e441527c None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.275839 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055abc2-a42d-4996-b9fe-0205e441527c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.276115 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055abc2-a42d-4996-b9fe-0205e441527c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.292242 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-60e20583-cee2-4c5e-85c2-3ea84f95ba4f None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:36.293453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-60e20583-cee2-4c5e-85c2-3ea84f95ba4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:36.296704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-0055abc2-a42d-4996-b9fe-0205e441527c None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:36.297898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0055abc2-a42d-4996-b9fe-0205e441527c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:36.301421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 782/1550] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:36.302651 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26800d51-3242-41c2-b595-75899678ab7e None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.303469 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26800d51-3242-41c2-b595-75899678ab7e None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.304128 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26800d51-3242-41c2-b595-75899678ab7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.313508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 769/1551] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:36.314975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bad3e0-feff-47f1-a56c-982410118d7f None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.315426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bad3e0-feff-47f1-a56c-982410118d7f None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.315862 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bad3e0-feff-47f1-a56c-982410118d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.326720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-26800d51-3242-41c2-b595-75899678ab7e None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:36.327907 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-26800d51-3242-41c2-b595-75899678ab7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:36.335797 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 783/1552] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:36.336552 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26800d51-3242-41c2-b595-75899678ab7e None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.337977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-05bad3e0-feff-47f1-a56c-982410118d7f None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:36.339161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-05bad3e0-feff-47f1-a56c-982410118d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:36.346994 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26800d51-3242-41c2-b595-75899678ab7e None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.347241 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 770/1553] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:36.348216 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b162de5-5b21-41aa-9e35-778699cb85d2 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.348513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b162de5-5b21-41aa-9e35-778699cb85d2 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.348772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b162de5-5b21-41aa-9e35-778699cb85d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.352388 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e32be6-9669-482b-b726-e4cdedd561aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.352977 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e32be6-9669-482b-b726-e4cdedd561aa admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.353329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e32be6-9669-482b-b726-e4cdedd561aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.353602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e32be6-9669-482b-b726-e4cdedd561aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.354221 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e32be6-9669-482b-b726-e4cdedd561aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.356080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e32be6-9669-482b-b726-e4cdedd561aa admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.369637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-9b162de5-5b21-41aa-9e35-778699cb85d2 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:36.370832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-9b162de5-5b21-41aa-9e35-778699cb85d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:36.379658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 771/1554] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:36.380439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b162de5-5b21-41aa-9e35-778699cb85d2 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.386737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 784/1555] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:36.388492 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadf5d8-0648-475a-8f0a-60ddc759b951 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.388786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadf5d8-0648-475a-8f0a-60ddc759b951 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.389061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadf5d8-0648-475a-8f0a-60ddc759b951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.389990 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b162de5-5b21-41aa-9e35-778699cb85d2 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.408992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-afadf5d8-0648-475a-8f0a-60ddc759b951 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:36.410307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-afadf5d8-0648-475a-8f0a-60ddc759b951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:36.447274 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b36c5d-cd64-4f5f-8f98-3a65d22073e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.447891 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b36c5d-cd64-4f5f-8f98-3a65d22073e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.448172 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b36c5d-cd64-4f5f-8f98-3a65d22073e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.448445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b36c5d-cd64-4f5f-8f98-3a65d22073e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.449089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b36c5d-cd64-4f5f-8f98-3a65d22073e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.450711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b36c5d-cd64-4f5f-8f98-3a65d22073e9 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.461154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 772/1556] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:36.462779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942b3bb-22eb-462e-ace9-a9f5a4e3fd80 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.463069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942b3bb-22eb-462e-ace9-a9f5a4e3fd80 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.463390 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942b3bb-22eb-462e-ace9-a9f5a4e3fd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.473154 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 785/1557] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:36.475297 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4606ee-24ba-4fc1-bddf-c79ef46f8795 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.475751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4606ee-24ba-4fc1-bddf-c79ef46f8795 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.476251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4606ee-24ba-4fc1-bddf-c79ef46f8795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.484701 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e942b3bb-22eb-462e-ace9-a9f5a4e3fd80 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:36.486089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e942b3bb-22eb-462e-ace9-a9f5a4e3fd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:36.495220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 773/1558] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:36.496251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a282c-8bc2-46cd-88e5-9947a544b0c2 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.496389 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-4a4606ee-24ba-4fc1-bddf-c79ef46f8795 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:36.496537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a282c-8bc2-46cd-88e5-9947a544b0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.496810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a282c-8bc2-46cd-88e5-9947a544b0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.497710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-4a4606ee-24ba-4fc1-bddf-c79ef46f8795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:36.507569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 786/1559] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:36.508432 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4606ee-24ba-4fc1-bddf-c79ef46f8795 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.511943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:36.518125 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4606ee-24ba-4fc1-bddf-c79ef46f8795 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.519157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-723a282c-8bc2-46cd-88e5-9947a544b0c2 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:36.520292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-723a282c-8bc2-46cd-88e5-9947a544b0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:36.522793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b654cf-77b2-4ba8-868f-2c93b5d6e720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.523390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b654cf-77b2-4ba8-868f-2c93b5d6e720 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.523670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b654cf-77b2-4ba8-868f-2c93b5d6e720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.523940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b654cf-77b2-4ba8-868f-2c93b5d6e720 admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.524571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b654cf-77b2-4ba8-868f-2c93b5d6e720 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:22:36.524884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b654cf-77b2-4ba8-868f-2c93b5d6e720 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.525919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: WARNING py.warnings [None req-a3b654cf-77b2-4ba8-868f-2c93b5d6e720 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 12:22:36.525919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: warnings.warn(msg) Sep 28 12:22:36.525919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: Sep 28 12:22:36.526793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b654cf-77b2-4ba8-868f-2c93b5d6e720 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.528601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 774/1560] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:36.533323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1095d3-952f-426f-91d8-9ce6f6c96aeb None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.534068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1095d3-952f-426f-91d8-9ce6f6c96aeb None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.534068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1095d3-952f-426f-91d8-9ce6f6c96aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.539508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 787/1561] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:22:36.540304 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b654cf-77b2-4ba8-868f-2c93b5d6e720 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.550797 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b654cf-77b2-4ba8-868f-2c93b5d6e720 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.555865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53e67f-554c-4436-a9d4-26a66c8fa9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.556458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53e67f-554c-4436-a9d4-26a66c8fa9ec admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.556811 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53e67f-554c-4436-a9d4-26a66c8fa9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.557089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53e67f-554c-4436-a9d4-26a66c8fa9ec admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.557822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53e67f-554c-4436-a9d4-26a66c8fa9ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.558493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-8c1095d3-952f-426f-91d8-9ce6f6c96aeb None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:36.559692 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8c1095d3-952f-426f-91d8-9ce6f6c96aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:36.559850 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53e67f-554c-4436-a9d4-26a66c8fa9ec admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.570594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:36.573202 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 775/1562] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:36.574078 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1095d3-952f-426f-91d8-9ce6f6c96aeb None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.586074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1095d3-952f-426f-91d8-9ce6f6c96aeb None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.589659 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 788/1563] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/users => generated 291 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:36.590575 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53e67f-554c-4436-a9d4-26a66c8fa9ec admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.590858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a0954-6e4f-4b38-acda-31c2b85ed2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.591813 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a0954-6e4f-4b38-acda-31c2b85ed2df admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.592300 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a0954-6e4f-4b38-acda-31c2b85ed2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.593083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a0954-6e4f-4b38-acda-31c2b85ed2df admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.594175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6a0954-6e4f-4b38-acda-31c2b85ed2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.597289 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6a0954-6e4f-4b38-acda-31c2b85ed2df admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.602443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53e67f-554c-4436-a9d4-26a66c8fa9ec admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.607531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569eac44-8ec4-4161-b534-d3b2bd12e05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.608138 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569eac44-8ec4-4161-b534-d3b2bd12e05c admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.608417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569eac44-8ec4-4161-b534-d3b2bd12e05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.608688 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569eac44-8ec4-4161-b534-d3b2bd12e05c admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.609374 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569eac44-8ec4-4161-b534-d3b2bd12e05c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:22:36.609676 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569eac44-8ec4-4161-b534-d3b2bd12e05c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.610182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 776/1564] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:36.611109 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a0954-6e4f-4b38-acda-31c2b85ed2df admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.611528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569eac44-8ec4-4161-b534-d3b2bd12e05c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.621767 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a0954-6e4f-4b38-acda-31c2b85ed2df admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.632569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 789/1565] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:22:36.632569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569eac44-8ec4-4161-b534-d3b2bd12e05c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.632853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e70d6-75af-4a0e-9694-59f4a16e6c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.632853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e70d6-75af-4a0e-9694-59f4a16e6c57 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.632853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e70d6-75af-4a0e-9694-59f4a16e6c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.632853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e70d6-75af-4a0e-9694-59f4a16e6c57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.633229 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e70d6-75af-4a0e-9694-59f4a16e6c57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:22:36.633229 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e70d6-75af-4a0e-9694-59f4a16e6c57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.633229 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e70d6-75af-4a0e-9694-59f4a16e6c57 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.636068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569eac44-8ec4-4161-b534-d3b2bd12e05c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.640636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ff800-b943-42c2-88e6-1ea8e8860b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.641246 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ff800-b943-42c2-88e6-1ea8e8860b22 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.641530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ff800-b943-42c2-88e6-1ea8e8860b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.641800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ff800-b943-42c2-88e6-1ea8e8860b22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.642425 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ff800-b943-42c2-88e6-1ea8e8860b22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:22:36.642729 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ff800-b943-42c2-88e6-1ea8e8860b22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.651341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 777/1566] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:22:36.651341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e70d6-75af-4a0e-9694-59f4a16e6c57 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.651620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ff800-b943-42c2-88e6-1ea8e8860b22 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.654371 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e70d6-75af-4a0e-9694-59f4a16e6c57 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.658641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 790/1567] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:22:36.659422 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950f01a-e170-4b88-a0c3-2ae320e06390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.659553 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ff800-b943-42c2-88e6-1ea8e8860b22 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.660028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950f01a-e170-4b88-a0c3-2ae320e06390 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.660307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950f01a-e170-4b88-a0c3-2ae320e06390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.660576 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950f01a-e170-4b88-a0c3-2ae320e06390 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.661242 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950f01a-e170-4b88-a0c3-2ae320e06390 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:22:36.661545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950f01a-e170-4b88-a0c3-2ae320e06390 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.663286 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950f01a-e170-4b88-a0c3-2ae320e06390 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.673443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ff800-b943-42c2-88e6-1ea8e8860b22 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.680641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 778/1568] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:22:36.680641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950f01a-e170-4b88-a0c3-2ae320e06390 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.680918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93e6cb-7eeb-4b4a-87d9-3950b7be440c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.680918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93e6cb-7eeb-4b4a-87d9-3950b7be440c admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.680918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93e6cb-7eeb-4b4a-87d9-3950b7be440c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.680918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93e6cb-7eeb-4b4a-87d9-3950b7be440c admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.681343 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93e6cb-7eeb-4b4a-87d9-3950b7be440c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.682430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93e6cb-7eeb-4b4a-87d9-3950b7be440c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.686847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950f01a-e170-4b88-a0c3-2ae320e06390 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.691357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db12869-c7a8-4e7e-b4b6-628006c190ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.691950 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db12869-c7a8-4e7e-b4b6-628006c190ee admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.692233 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db12869-c7a8-4e7e-b4b6-628006c190ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.692538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db12869-c7a8-4e7e-b4b6-628006c190ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.693195 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db12869-c7a8-4e7e-b4b6-628006c190ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:22:36.694001 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db12869-c7a8-4e7e-b4b6-628006c190ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.695693 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db12869-c7a8-4e7e-b4b6-628006c190ee admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.707830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 779/1569] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:22:36.708626 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db12869-c7a8-4e7e-b4b6-628006c190ee admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.715470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 791/1570] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:36.715470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93e6cb-7eeb-4b4a-87d9-3950b7be440c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.721443 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db12869-c7a8-4e7e-b4b6-628006c190ee admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.777652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93e6cb-7eeb-4b4a-87d9-3950b7be440c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.779578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634cfc14-8200-4e16-8365-94e4274bc66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.780215 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cfc14-8200-4e16-8365-94e4274bc66f admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.780499 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cfc14-8200-4e16-8365-94e4274bc66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.780770 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cfc14-8200-4e16-8365-94e4274bc66f admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.781441 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634cfc14-8200-4e16-8365-94e4274bc66f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.782759 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38588fa-a471-4fe3-9853-4e238c101118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.783030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634cfc14-8200-4e16-8365-94e4274bc66f admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.783427 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38588fa-a471-4fe3-9853-4e238c101118 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.783712 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38588fa-a471-4fe3-9853-4e238c101118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.783987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38588fa-a471-4fe3-9853-4e238c101118 admin admin] PATH_INFO: `/v3/projects/a396283cd8314a74bfd5e411c246cc96/users/4d7b7c68a699443aa5c0859094c265ec/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.792993 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 780/1571] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:36.794638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634cfc14-8200-4e16-8365-94e4274bc66f admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.801974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38588fa-a471-4fe3-9853-4e238c101118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a396283cd8314a74bfd5e411c246cc96, user_id=4d7b7c68a699443aa5c0859094c265ec, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.804734 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38588fa-a471-4fe3-9853-4e238c101118 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.805196 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634cfc14-8200-4e16-8365-94e4274bc66f admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.810232 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec82c7-0b20-462b-8bf7-5aa5fcaa5015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.810959 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec82c7-0b20-462b-8bf7-5aa5fcaa5015 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.811246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec82c7-0b20-462b-8bf7-5aa5fcaa5015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.811526 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec82c7-0b20-462b-8bf7-5aa5fcaa5015 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.812295 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ec82c7-0b20-462b-8bf7-5aa5fcaa5015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.814371 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ec82c7-0b20-462b-8bf7-5aa5fcaa5015 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.818281 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 792/1572] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:36 2022] PUT /identity/v3/projects/a396283cd8314a74bfd5e411c246cc96/users/4d7b7c68a699443aa5c0859094c265ec/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:36.819769 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38588fa-a471-4fe3-9853-4e238c101118 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.845847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 781/1573] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:36.846825 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec82c7-0b20-462b-8bf7-5aa5fcaa5015 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.885652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38588fa-a471-4fe3-9853-4e238c101118 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.908791 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec82c7-0b20-462b-8bf7-5aa5fcaa5015 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:36.914210 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f5a13-836a-40ca-ae68-7a6e272d93fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.914821 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f5a13-836a-40ca-ae68-7a6e272d93fb admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.915264 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f5a13-836a-40ca-ae68-7a6e272d93fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.915510 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f5a13-836a-40ca-ae68-7a6e272d93fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.916237 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f5a13-836a-40ca-ae68-7a6e272d93fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.917059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:36.918582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f5a13-836a-40ca-ae68-7a6e272d93fb admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.944766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd16d71-61eb-4a6d-8941-3ac87533e314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:36.945560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd16d71-61eb-4a6d-8941-3ac87533e314 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:36.945836 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd16d71-61eb-4a6d-8941-3ac87533e314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:36.946102 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd16d71-61eb-4a6d-8941-3ac87533e314 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:36.946533 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 782/1574] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:36.946736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd16d71-61eb-4a6d-8941-3ac87533e314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:36.948588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd16d71-61eb-4a6d-8941-3ac87533e314 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:36.950541 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f5a13-836a-40ca-ae68-7a6e272d93fb admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:36.954684 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:36.980641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 793/1575] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/projects => generated 403 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:36.980641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd16d71-61eb-4a6d-8941-3ac87533e314 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.014100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f5a13-836a-40ca-ae68-7a6e272d93fb admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.041597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd16d71-61eb-4a6d-8941-3ac87533e314 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.046303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f81116-c91b-4ec2-af58-565b2c726245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.047025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f81116-c91b-4ec2-af58-565b2c726245 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.047373 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f81116-c91b-4ec2-af58-565b2c726245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.047710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f81116-c91b-4ec2-af58-565b2c726245 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.048333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f81116-c91b-4ec2-af58-565b2c726245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.050338 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f81116-c91b-4ec2-af58-565b2c726245 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.070773 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4700a2d-d683-4709-82b5-04973c521b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.071390 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4700a2d-d683-4709-82b5-04973c521b05 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.071670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4700a2d-d683-4709-82b5-04973c521b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.071953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4700a2d-d683-4709-82b5-04973c521b05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.072074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 794/1576] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:37.072626 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4700a2d-d683-4709-82b5-04973c521b05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.072965 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f81116-c91b-4ec2-af58-565b2c726245 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.074645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4700a2d-d683-4709-82b5-04973c521b05 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.082438 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f81116-c91b-4ec2-af58-565b2c726245 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.087803 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150f33a-794e-4b57-9623-4c62bad587a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.088385 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150f33a-794e-4b57-9623-4c62bad587a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.088665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150f33a-794e-4b57-9623-4c62bad587a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.088938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150f33a-794e-4b57-9623-4c62bad587a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.089586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150f33a-794e-4b57-9623-4c62bad587a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.091461 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150f33a-794e-4b57-9623-4c62bad587a0 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.101802 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 783/1577] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:36 2022] POST /identity/v3/projects => generated 367 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:37.102771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4700a2d-d683-4709-82b5-04973c521b05 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.122715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 795/1578] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:37 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:37.124132 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c45cf6-cd1c-4c22-b26e-60672ed4bd38 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.124413 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c45cf6-cd1c-4c22-b26e-60672ed4bd38 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.124685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c45cf6-cd1c-4c22-b26e-60672ed4bd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.143081 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-55c45cf6-cd1c-4c22-b26e-60672ed4bd38 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:37.144370 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-55c45cf6-cd1c-4c22-b26e-60672ed4bd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:37.170684 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4700a2d-d683-4709-82b5-04973c521b05 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.204318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 796/1579] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:37.205321 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c45cf6-cd1c-4c22-b26e-60672ed4bd38 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.217067 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:37.219142 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c45cf6-cd1c-4c22-b26e-60672ed4bd38 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.225501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8531c-22b5-4acb-b199-f7e137d0ab74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.226447 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8531c-22b5-4acb-b199-f7e137d0ab74 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.226733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8531c-22b5-4acb-b199-f7e137d0ab74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.227007 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8531c-22b5-4acb-b199-f7e137d0ab74 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.227634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8531c-22b5-4acb-b199-f7e137d0ab74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.229574 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8531c-22b5-4acb-b199-f7e137d0ab74 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.237503 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd21da3-90c4-490b-a427-58bc2e13bb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.238121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd21da3-90c4-490b-a427-58bc2e13bb0a admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.238401 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd21da3-90c4-490b-a427-58bc2e13bb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.239038 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd21da3-90c4-490b-a427-58bc2e13bb0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.239339 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd21da3-90c4-490b-a427-58bc2e13bb0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.241312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd21da3-90c4-490b-a427-58bc2e13bb0a admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.251104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 784/1580] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:37.251363 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 797/1581] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:37 2022] POST /identity/v3/users => generated 287 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:37.251957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd21da3-90c4-490b-a427-58bc2e13bb0a admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.252166 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8531c-22b5-4acb-b199-f7e137d0ab74 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.262970 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd21da3-90c4-490b-a427-58bc2e13bb0a admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.266917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8531c-22b5-4acb-b199-f7e137d0ab74 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.267776 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03e960-255c-4086-adfd-ba77a95771b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.268400 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03e960-255c-4086-adfd-ba77a95771b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.268697 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03e960-255c-4086-adfd-ba77a95771b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.268949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03e960-255c-4086-adfd-ba77a95771b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.269847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03e960-255c-4086-adfd-ba77a95771b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.272379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6856cca-3e7c-4e8e-9c27-f44e621c7b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.272541 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03e960-255c-4086-adfd-ba77a95771b1 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.273046 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6856cca-3e7c-4e8e-9c27-f44e621c7b49 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.273367 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6856cca-3e7c-4e8e-9c27-f44e621c7b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.273648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6856cca-3e7c-4e8e-9c27-f44e621c7b49 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.274280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6856cca-3e7c-4e8e-9c27-f44e621c7b49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.276211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6856cca-3e7c-4e8e-9c27-f44e621c7b49 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.285107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:37.298739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 785/1582] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:37 2022] POST /identity/v3/users => generated 293 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:37.298739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03e960-255c-4086-adfd-ba77a95771b1 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.298986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 798/1583] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:37 2022] POST /identity/v3/users => generated 302 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:37.299449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6856cca-3e7c-4e8e-9c27-f44e621c7b49 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.308026 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03e960-255c-4086-adfd-ba77a95771b1 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.309735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6856cca-3e7c-4e8e-9c27-f44e621c7b49 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.312683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea26a8b-5e6f-4c92-ac32-f20fbcc69edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.313316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea26a8b-5e6f-4c92-ac32-f20fbcc69edb admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.313592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea26a8b-5e6f-4c92-ac32-f20fbcc69edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.314142 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea26a8b-5e6f-4c92-ac32-f20fbcc69edb admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.314412 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25dc6c-0983-4bd4-96a4-ca57a7148d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.314774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea26a8b-5e6f-4c92-ac32-f20fbcc69edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.315004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25dc6c-0983-4bd4-96a4-ca57a7148d0d admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.315285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25dc6c-0983-4bd4-96a4-ca57a7148d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.315561 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c25dc6c-0983-4bd4-96a4-ca57a7148d0d admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.316181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25dc6c-0983-4bd4-96a4-ca57a7148d0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.316552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea26a8b-5e6f-4c92-ac32-f20fbcc69edb admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.318093 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c25dc6c-0983-4bd4-96a4-ca57a7148d0d admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.328859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 786/1584] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:37.329685 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea26a8b-5e6f-4c92-ac32-f20fbcc69edb admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.340436 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea26a8b-5e6f-4c92-ac32-f20fbcc69edb admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.346951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 799/1585] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:37 2022] POST /identity/v3/users => generated 284 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:37.346951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25dc6c-0983-4bd4-96a4-ca57a7148d0d admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.347179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2b84d-7620-4124-b350-c6b3e7083ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.347179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2b84d-7620-4124-b350-c6b3e7083ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.347179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2b84d-7620-4124-b350-c6b3e7083ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.347179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2b84d-7620-4124-b350-c6b3e7083ef0 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.347544 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2b84d-7620-4124-b350-c6b3e7083ef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.348598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2b84d-7620-4124-b350-c6b3e7083ef0 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.374084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 787/1586] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:37 2022] POST /identity/v3/users => generated 287 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:37.374084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2b84d-7620-4124-b350-c6b3e7083ef0 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.382805 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2b84d-7620-4124-b350-c6b3e7083ef0 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.387421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bda12-cd1e-406f-a34f-41812057a5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.388063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bda12-cd1e-406f-a34f-41812057a5b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.388348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bda12-cd1e-406f-a34f-41812057a5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.388618 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bda12-cd1e-406f-a34f-41812057a5b7 admin admin] PATH_INFO: `/v3/projects/b9be06233afe44b2a7bc6da9e971056e/users/7c5c1a11cd394a559d85a162cd82ae9c/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.402708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c25dc6c-0983-4bd4-96a4-ca57a7148d0d admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.403527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bda12-cd1e-406f-a34f-41812057a5b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9be06233afe44b2a7bc6da9e971056e, user_id=7c5c1a11cd394a559d85a162cd82ae9c, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.404567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: WARNING py.warnings [None req-f69bda12-cd1e-406f-a34f-41812057a5b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 12:22:37.404567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: warnings.warn(msg) Sep 28 12:22:37.404567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: Sep 28 12:22:37.406401 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e27792-d7b2-41b3-89c8-af04ef0a4d3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.406527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bda12-cd1e-406f-a34f-41812057a5b7 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.407004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e27792-d7b2-41b3-89c8-af04ef0a4d3e service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.407278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e27792-d7b2-41b3-89c8-af04ef0a4d3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.407546 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e27792-d7b2-41b3-89c8-af04ef0a4d3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.416669 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e27792-d7b2-41b3-89c8-af04ef0a4d3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.418108 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e27792-d7b2-41b3-89c8-af04ef0a4d3e service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.419045 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 788/1587] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:37 2022] PUT /identity/v3/projects/b9be06233afe44b2a7bc6da9e971056e/users/7c5c1a11cd394a559d85a162cd82ae9c/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:37.429366 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bda12-cd1e-406f-a34f-41812057a5b7 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.429531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 800/1588] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:37.429531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e27792-d7b2-41b3-89c8-af04ef0a4d3e service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.438794 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e27792-d7b2-41b3-89c8-af04ef0a4d3e service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.499873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bda12-cd1e-406f-a34f-41812057a5b7 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.505157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560d2b5-2006-486a-af30-866a99964955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.505511 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf4f38-c684-4f78-9eda-492aa5d494d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.505906 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560d2b5-2006-486a-af30-866a99964955 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.506963 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560d2b5-2006-486a-af30-866a99964955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.507077 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf4f38-c684-4f78-9eda-492aa5d494d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.507178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560d2b5-2006-486a-af30-866a99964955 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.508043 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560d2b5-2006-486a-af30-866a99964955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.508174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf4f38-c684-4f78-9eda-492aa5d494d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.508399 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf4f38-c684-4f78-9eda-492aa5d494d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.509377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf4f38-c684-4f78-9eda-492aa5d494d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.509914 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560d2b5-2006-486a-af30-866a99964955 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.510800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf4f38-c684-4f78-9eda-492aa5d494d2 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.523413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 789/1589] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:37.524767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 801/1590] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:37.525718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf4f38-c684-4f78-9eda-492aa5d494d2 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.526261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560d2b5-2006-486a-af30-866a99964955 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.537952 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560d2b5-2006-486a-af30-866a99964955 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.538162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf4f38-c684-4f78-9eda-492aa5d494d2 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.542889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c89b2-53f9-4778-aaa5-de14702d9d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.543032 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b7716-daca-4643-8879-b3df983083eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.543523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c89b2-53f9-4778-aaa5-de14702d9d98 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.543643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b7716-daca-4643-8879-b3df983083eb admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.543812 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c89b2-53f9-4778-aaa5-de14702d9d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.543920 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b7716-daca-4643-8879-b3df983083eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.544108 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b7716-daca-4643-8879-b3df983083eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.544225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c89b2-53f9-4778-aaa5-de14702d9d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.544698 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c89b2-53f9-4778-aaa5-de14702d9d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.544822 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b7716-daca-4643-8879-b3df983083eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.546322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c89b2-53f9-4778-aaa5-de14702d9d98 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.546429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b7716-daca-4643-8879-b3df983083eb admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.557154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 790/1591] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:37.557315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 802/1592] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:37.558204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c89b2-53f9-4778-aaa5-de14702d9d98 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.562419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b7716-daca-4643-8879-b3df983083eb admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.569621 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c89b2-53f9-4778-aaa5-de14702d9d98 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.574421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce377ff9-a891-475f-8618-07da1bd8edeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.574997 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b7716-daca-4643-8879-b3df983083eb admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.575127 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce377ff9-a891-475f-8618-07da1bd8edeb admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.575294 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce377ff9-a891-475f-8618-07da1bd8edeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.575558 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce377ff9-a891-475f-8618-07da1bd8edeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.576191 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce377ff9-a891-475f-8618-07da1bd8edeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.577855 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce377ff9-a891-475f-8618-07da1bd8edeb admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.580284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d7d79-f8b4-4249-ad25-29e75c3f8a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.581102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d7d79-f8b4-4249-ad25-29e75c3f8a7d admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.581599 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d7d79-f8b4-4249-ad25-29e75c3f8a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.581974 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d7d79-f8b4-4249-ad25-29e75c3f8a7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.582714 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d7d79-f8b4-4249-ad25-29e75c3f8a7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.585100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d7d79-f8b4-4249-ad25-29e75c3f8a7d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.589706 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 803/1593] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:37.590762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87858231-a2fe-42f2-8757-c439d081130f None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.591047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87858231-a2fe-42f2-8757-c439d081130f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.591318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87858231-a2fe-42f2-8757-c439d081130f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.600900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 791/1594] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:37.605300 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d7d79-f8b4-4249-ad25-29e75c3f8a7d admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.616386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d7d79-f8b4-4249-ad25-29e75c3f8a7d admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.621190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f060c1-cd1d-43cc-94df-f034c17f4035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.621901 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f060c1-cd1d-43cc-94df-f034c17f4035 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.622251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f060c1-cd1d-43cc-94df-f034c17f4035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.622591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f060c1-cd1d-43cc-94df-f034c17f4035 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.622732 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-87858231-a2fe-42f2-8757-c439d081130f None None] MFA Rules not processed for user `7c5c1a11cd394a559d85a162cd82ae9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:37.623357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f060c1-cd1d-43cc-94df-f034c17f4035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.623999 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-87858231-a2fe-42f2-8757-c439d081130f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:37.625087 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f060c1-cd1d-43cc-94df-f034c17f4035 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.636251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 792/1595] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:37.637035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f060c1-cd1d-43cc-94df-f034c17f4035 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.652112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f060c1-cd1d-43cc-94df-f034c17f4035 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.656048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bee11-c4e7-417f-a690-024641651204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.656718 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bee11-c4e7-417f-a690-024641651204 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.657071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bee11-c4e7-417f-a690-024641651204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.657300 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bee11-c4e7-417f-a690-024641651204 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.657887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bee11-c4e7-417f-a690-024641651204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.659589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bee11-c4e7-417f-a690-024641651204 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.671847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 793/1596] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:37.672998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bee11-c4e7-417f-a690-024641651204 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.685490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bee11-c4e7-417f-a690-024641651204 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.690243 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a275502-78df-43bd-8a4f-1743951792d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.690847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a275502-78df-43bd-8a4f-1743951792d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.691077 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:37.691205 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a275502-78df-43bd-8a4f-1743951792d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.691374 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a275502-78df-43bd-8a4f-1743951792d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.692012 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a275502-78df-43bd-8a4f-1743951792d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.694484 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a275502-78df-43bd-8a4f-1743951792d4 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.706946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 794/1597] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:37.707741 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a275502-78df-43bd-8a4f-1743951792d4 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.720472 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a275502-78df-43bd-8a4f-1743951792d4 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.725874 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 804/1598] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:37.726031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98bd6e-89e1-4f94-91c5-89f90e86f292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.726477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98bd6e-89e1-4f94-91c5-89f90e86f292 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.726833 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98bd6e-89e1-4f94-91c5-89f90e86f292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.727089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87858231-a2fe-42f2-8757-c439d081130f None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.727215 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c98bd6e-89e1-4f94-91c5-89f90e86f292 admin admin] PATH_INFO: `/v3/projects/3afaa30b2f454bc0bf1f23ece426339b/users/95575c3357d4413298c3fa08709c5037/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.732300 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:37.738671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87858231-a2fe-42f2-8757-c439d081130f None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.745849 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f4a2c-4631-4467-8438-02cba7b23933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.745849 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f4a2c-4631-4467-8438-02cba7b23933 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.746262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f4a2c-4631-4467-8438-02cba7b23933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.746630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f4a2c-4631-4467-8438-02cba7b23933 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.746768 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c98bd6e-89e1-4f94-91c5-89f90e86f292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3afaa30b2f454bc0bf1f23ece426339b, user_id=95575c3357d4413298c3fa08709c5037, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.747427 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f4a2c-4631-4467-8438-02cba7b23933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.749271 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f4a2c-4631-4467-8438-02cba7b23933 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.749853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c98bd6e-89e1-4f94-91c5-89f90e86f292 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.761470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 805/1599] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:37.764017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 795/1600] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:37 2022] PUT /identity/v3/projects/3afaa30b2f454bc0bf1f23ece426339b/users/95575c3357d4413298c3fa08709c5037/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:37.766081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98bd6e-89e1-4f94-91c5-89f90e86f292 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.766301 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f4a2c-4631-4467-8438-02cba7b23933 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.776628 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f4a2c-4631-4467-8438-02cba7b23933 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.844032 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553cafb-7f1e-49bb-b769-b783cc6f42a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.844867 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553cafb-7f1e-49bb-b769-b783cc6f42a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.845051 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553cafb-7f1e-49bb-b769-b783cc6f42a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.845381 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553cafb-7f1e-49bb-b769-b783cc6f42a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.846022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553cafb-7f1e-49bb-b769-b783cc6f42a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.847629 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553cafb-7f1e-49bb-b769-b783cc6f42a8 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.862140 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 806/1601] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:37.862140 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553cafb-7f1e-49bb-b769-b783cc6f42a8 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.868629 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c98bd6e-89e1-4f94-91c5-89f90e86f292 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.874267 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553cafb-7f1e-49bb-b769-b783cc6f42a8 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:37.875452 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9bd89-ea7a-4a0f-8816-eb70015c01c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.876181 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9bd89-ea7a-4a0f-8816-eb70015c01c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.876467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9bd89-ea7a-4a0f-8816-eb70015c01c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.876800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9bd89-ea7a-4a0f-8816-eb70015c01c1 admin admin] PATH_INFO: `/v3/projects/f19c33b4bea04ee582a9424c2ddfdade/users/0e4b2eb9978342ab80e698151fdcf049/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.880757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31598a07-c500-425f-a309-8575d547f01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:37.882539 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31598a07-c500-425f-a309-8575d547f01d admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:37.882932 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31598a07-c500-425f-a309-8575d547f01d admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:37.883405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31598a07-c500-425f-a309-8575d547f01d admin admin] PATH_INFO: `/v3/projects/8476c581992e465fbd1f81526088b5eb/users/d1d5082d04b843d0a31dda851bfff1d4/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:37.902757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9bd89-ea7a-4a0f-8816-eb70015c01c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f19c33b4bea04ee582a9424c2ddfdade, user_id=0e4b2eb9978342ab80e698151fdcf049, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.904662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31598a07-c500-425f-a309-8575d547f01d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8476c581992e465fbd1f81526088b5eb, user_id=d1d5082d04b843d0a31dda851bfff1d4, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:37.906147 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9bd89-ea7a-4a0f-8816-eb70015c01c1 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.908453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31598a07-c500-425f-a309-8575d547f01d admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:37.921674 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 796/1602] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:37 2022] PUT /identity/v3/projects/f19c33b4bea04ee582a9424c2ddfdade/users/0e4b2eb9978342ab80e698151fdcf049/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:37.933403 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 807/1603] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:37 2022] PUT /identity/v3/projects/8476c581992e465fbd1f81526088b5eb/users/d1d5082d04b843d0a31dda851bfff1d4/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:37.933403 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31598a07-c500-425f-a309-8575d547f01d admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:37.933646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9bd89-ea7a-4a0f-8816-eb70015c01c1 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:38.004942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31598a07-c500-425f-a309-8575d547f01d admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:38.005280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9bd89-ea7a-4a0f-8816-eb70015c01c1 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:38.011693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446af41-1548-44ef-a7eb-bf1dae305cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:38.011881 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fb6e4-ad02-46c1-b4b3-e5450d56dbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:38.012362 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446af41-1548-44ef-a7eb-bf1dae305cd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.012474 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fb6e4-ad02-46c1-b4b3-e5450d56dbdc admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.012639 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446af41-1548-44ef-a7eb-bf1dae305cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.012882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fb6e4-ad02-46c1-b4b3-e5450d56dbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.012983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446af41-1548-44ef-a7eb-bf1dae305cd4 admin admin] PATH_INFO: `/v3/projects/8f9ceb72510849c9a1cce4cde67d06d2/users/8429a502bca744fc9f5d5701fe9c766c/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.013243 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fb6e4-ad02-46c1-b4b3-e5450d56dbdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.014620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fb6e4-ad02-46c1-b4b3-e5450d56dbdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:38.017450 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fb6e4-ad02-46c1-b4b3-e5450d56dbdc admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:38.028945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446af41-1548-44ef-a7eb-bf1dae305cd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9ceb72510849c9a1cce4cde67d06d2, user_id=8429a502bca744fc9f5d5701fe9c766c, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:38.031617 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446af41-1548-44ef-a7eb-bf1dae305cd4 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:38.049371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 808/1604] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:37 2022] PUT /identity/v3/projects/8f9ceb72510849c9a1cce4cde67d06d2/users/8429a502bca744fc9f5d5701fe9c766c/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:38.049371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446af41-1548-44ef-a7eb-bf1dae305cd4 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:38.054220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 797/1605] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:37 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:38.055790 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea955c-0eef-44b2-b66c-78803a2454cb None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.056142 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea955c-0eef-44b2-b66c-78803a2454cb None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.056574 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea955c-0eef-44b2-b66c-78803a2454cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.078725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-8eea955c-0eef-44b2-b66c-78803a2454cb None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:38.079765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:38.079889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8eea955c-0eef-44b2-b66c-78803a2454cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:38.125880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446af41-1548-44ef-a7eb-bf1dae305cd4 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:38.139524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:38.154987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 798/1606] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:38.155942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea955c-0eef-44b2-b66c-78803a2454cb None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:38.169512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea955c-0eef-44b2-b66c-78803a2454cb None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:38.177777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f09193-42e7-416c-9d7e-db0e4b1eb0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:38.179473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f09193-42e7-416c-9d7e-db0e4b1eb0db admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.180555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f09193-42e7-416c-9d7e-db0e4b1eb0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.180988 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f09193-42e7-416c-9d7e-db0e4b1eb0db admin admin] PATH_INFO: `/v3/projects/bb55541c8a31486494dac11aa5c84e37/users/98290356ebed4635a170716391152f57/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.200706 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6890e-43f6-4561-9ec6-0b5248d968ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:38.201433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6890e-43f6-4561-9ec6-0b5248d968ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.201808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6890e-43f6-4561-9ec6-0b5248d968ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.202194 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6890e-43f6-4561-9ec6-0b5248d968ca admin admin] PATH_INFO: `/v3/projects/9cdf002f6eec42199d3619b26dcb7384/users/bc4f453670c846469b313199f6056839/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.219744 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6890e-43f6-4561-9ec6-0b5248d968ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cdf002f6eec42199d3619b26dcb7384, user_id=bc4f453670c846469b313199f6056839, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:38.220457 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f09193-42e7-416c-9d7e-db0e4b1eb0db admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb55541c8a31486494dac11aa5c84e37, user_id=98290356ebed4635a170716391152f57, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:38.222794 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6890e-43f6-4561-9ec6-0b5248d968ca admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:38.223548 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f09193-42e7-416c-9d7e-db0e4b1eb0db admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:38.243909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 799/1607] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:38 2022] PUT /identity/v3/projects/bb55541c8a31486494dac11aa5c84e37/users/98290356ebed4635a170716391152f57/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:38.245204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b993e-2bdb-4d01-90fc-f3d329ebe261 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.245553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b993e-2bdb-4d01-90fc-f3d329ebe261 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.245826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b993e-2bdb-4d01-90fc-f3d329ebe261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.257491 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 809/1608] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:38 2022] PUT /identity/v3/projects/9cdf002f6eec42199d3619b26dcb7384/users/bc4f453670c846469b313199f6056839/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:38.261821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538f64a-b41e-427c-91a5-5fd9693ce9e8 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.261821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538f64a-b41e-427c-91a5-5fd9693ce9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.261821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538f64a-b41e-427c-91a5-5fd9693ce9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.291903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-2e8b993e-2bdb-4d01-90fc-f3d329ebe261 None None] MFA Rules not processed for user `0e4b2eb9978342ab80e698151fdcf049`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:38.293576 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2e8b993e-2bdb-4d01-90fc-f3d329ebe261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:38.316217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-1538f64a-b41e-427c-91a5-5fd9693ce9e8 None None] MFA Rules not processed for user `d1d5082d04b843d0a31dda851bfff1d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:38.317916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-1538f64a-b41e-427c-91a5-5fd9693ce9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:38.378328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 800/1609] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:38.384516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2416db7-faa0-42e3-b6e5-230792daa8e6 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.384874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2416db7-faa0-42e3-b6e5-230792daa8e6 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.385254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2416db7-faa0-42e3-b6e5-230792daa8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.414728 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-b2416db7-faa0-42e3-b6e5-230792daa8e6 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:38.417463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b2416db7-faa0-42e3-b6e5-230792daa8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:38.452007 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 810/1610] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:38.453109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538f64a-b41e-427c-91a5-5fd9693ce9e8 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:38.510774 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:38.519257 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 801/1611] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:38.520173 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2416db7-faa0-42e3-b6e5-230792daa8e6 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:38.548036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538f64a-b41e-427c-91a5-5fd9693ce9e8 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:38.553891 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb6640-626e-4ab7-b5d2-cb8b669f8c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:38.554633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb6640-626e-4ab7-b5d2-cb8b669f8c75 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.554978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb6640-626e-4ab7-b5d2-cb8b669f8c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.555354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb6640-626e-4ab7-b5d2-cb8b669f8c75 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.556054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb6640-626e-4ab7-b5d2-cb8b669f8c75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:38.558990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb6640-626e-4ab7-b5d2-cb8b669f8c75 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:38.591759 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 811/1612] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:38 2022] POST /identity/v3/users => generated 297 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:38.591759 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110c837-3b2d-4658-9150-a5acae737486 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.591759 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110c837-3b2d-4658-9150-a5acae737486 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.591759 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110c837-3b2d-4658-9150-a5acae737486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.594669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2416db7-faa0-42e3-b6e5-230792daa8e6 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:38.601356 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab4786-0192-47e6-9967-b8c62f606a2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:38.602010 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab4786-0192-47e6-9967-b8c62f606a2b service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.602287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab4786-0192-47e6-9967-b8c62f606a2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.602557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab4786-0192-47e6-9967-b8c62f606a2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.621943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab4786-0192-47e6-9967-b8c62f606a2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:38.625022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab4786-0192-47e6-9967-b8c62f606a2b service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:38.625440 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-f110c837-3b2d-4658-9150-a5acae737486 None None] MFA Rules not processed for user `98290356ebed4635a170716391152f57`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:38.626681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-f110c837-3b2d-4658-9150-a5acae737486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:38.642488 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 802/1613] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:38.644845 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812940d-e7f5-4aad-9fe6-c526474b41f5 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.645378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812940d-e7f5-4aad-9fe6-c526474b41f5 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.646247 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812940d-e7f5-4aad-9fe6-c526474b41f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.659455 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:38.699935 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-6812940d-e7f5-4aad-9fe6-c526474b41f5 None None] MFA Rules not processed for user `bc4f453670c846469b313199f6056839`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:38.701411 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-6812940d-e7f5-4aad-9fe6-c526474b41f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:38.713835 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 812/1614] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:38.715190 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67bdedd-f82b-4795-bb1f-63f99793a1ca None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.715498 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67bdedd-f82b-4795-bb1f-63f99793a1ca None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.715795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67bdedd-f82b-4795-bb1f-63f99793a1ca None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.718345 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 813/1615] 10.222.0.60 () {54 vars in 961 bytes} [Wed Sep 28 12:22:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:22:38.719175 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67bdedd-f82b-4795-bb1f-63f99793a1ca None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:38.749758 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67bdedd-f82b-4795-bb1f-63f99793a1ca None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:38.763737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7553f-39e2-4de2-9639-4402a9765b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:38.768328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7553f-39e2-4de2-9639-4402a9765b24 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.770357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7553f-39e2-4de2-9639-4402a9765b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.770747 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7553f-39e2-4de2-9639-4402a9765b24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.771717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7553f-39e2-4de2-9639-4402a9765b24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:38.776743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7553f-39e2-4de2-9639-4402a9765b24 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:38.815592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 803/1616] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:38.817497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812940d-e7f5-4aad-9fe6-c526474b41f5 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:38.820690 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-6812940d-e7f5-4aad-9fe6-c526474b41f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:38.833576 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812940d-e7f5-4aad-9fe6-c526474b41f5 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:38.838205 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbab54e-0cc2-4d84-8ce0-29eabb6a7097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:38.838817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbab54e-0cc2-4d84-8ce0-29eabb6a7097 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.839095 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbab54e-0cc2-4d84-8ce0-29eabb6a7097 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.839390 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbab54e-0cc2-4d84-8ce0-29eabb6a7097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.839918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 814/1617] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:38 2022] POST /identity/v3/projects => generated 403 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:38.841001 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7553f-39e2-4de2-9639-4402a9765b24 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:38.854722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbab54e-0cc2-4d84-8ce0-29eabb6a7097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:38.855951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbab54e-0cc2-4d84-8ce0-29eabb6a7097 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:38.882264 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 804/1618] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:38.888369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a53f2e-a3cc-46f8-829d-19961aafa4b1 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.888731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a53f2e-a3cc-46f8-829d-19961aafa4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.889573 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a53f2e-a3cc-46f8-829d-19961aafa4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.936815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7553f-39e2-4de2-9639-4402a9765b24 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:38.942693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0912c75-2976-494e-aa55-d6bdada46744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:38.944251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0912c75-2976-494e-aa55-d6bdada46744 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.945644 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0912c75-2976-494e-aa55-d6bdada46744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.946118 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0912c75-2976-494e-aa55-d6bdada46744 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.946983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0912c75-2976-494e-aa55-d6bdada46744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:38.949082 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0912c75-2976-494e-aa55-d6bdada46744 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:38.966965 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 815/1619] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:38.968404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9013280-4b90-4cca-a66e-039fa089c52b None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:38.969074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9013280-4b90-4cca-a66e-039fa089c52b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:38.969476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9013280-4b90-4cca-a66e-039fa089c52b None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:38.978043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 816/1620] 10.222.0.60 () {54 vars in 961 bytes} [Wed Sep 28 12:22:38 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:22:38.979201 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9013280-4b90-4cca-a66e-039fa089c52b None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:38.986555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-47a53f2e-a3cc-46f8-829d-19961aafa4b1 None None] MFA Rules not processed for user `982b82e8ac424318a934f9149fb91706`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:38.989168 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-47a53f2e-a3cc-46f8-829d-19961aafa4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:38.998935 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9013280-4b90-4cca-a66e-039fa089c52b None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:39.006586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0477f-5f00-4cc8-bd64-3fb922e40647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:39.008265 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0477f-5f00-4cc8-bd64-3fb922e40647 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:39.008725 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0477f-5f00-4cc8-bd64-3fb922e40647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:39.009420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0477f-5f00-4cc8-bd64-3fb922e40647 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:39.010275 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0477f-5f00-4cc8-bd64-3fb922e40647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:39.014363 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0477f-5f00-4cc8-bd64-3fb922e40647 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:39.068537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 817/1621] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:38 2022] POST /identity/v3/projects => generated 369 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:39.068537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0477f-5f00-4cc8-bd64-3fb922e40647 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:39.101314 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:39.159301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 805/1622] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Sep 28 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:39.160370 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a53f2e-a3cc-46f8-829d-19961aafa4b1 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:39.164702 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0477f-5f00-4cc8-bd64-3fb922e40647 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:39.171984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98be9a-4c50-44bb-b7b8-182d262aa91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:39.177269 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98be9a-4c50-44bb-b7b8-182d262aa91e admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:39.178172 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98be9a-4c50-44bb-b7b8-182d262aa91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:39.178652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98be9a-4c50-44bb-b7b8-182d262aa91e admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:39.179481 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98be9a-4c50-44bb-b7b8-182d262aa91e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:39.179633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a53f2e-a3cc-46f8-829d-19961aafa4b1 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:39.181907 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98be9a-4c50-44bb-b7b8-182d262aa91e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:39.186360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd90b40-d47a-45ee-9c03-ca53564acef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:39.187279 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd90b40-d47a-45ee-9c03-ca53564acef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:39.187547 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd90b40-d47a-45ee-9c03-ca53564acef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:39.187823 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd90b40-d47a-45ee-9c03-ca53564acef5 admin admin] PATH_INFO: `/v3/projects/6d5e228f10a440e784e7439dd6a1bfc8/users/98f76e47ee764cf094b372a5d8792f1f/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:39.213125 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd90b40-d47a-45ee-9c03-ca53564acef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d5e228f10a440e784e7439dd6a1bfc8, user_id=98f76e47ee764cf094b372a5d8792f1f, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:39.216624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd90b40-d47a-45ee-9c03-ca53564acef5 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:39.225544 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 818/1623] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:39 2022] POST /identity/v3/users => generated 308 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:39.227258 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac7d5a-01dc-445e-a73a-ca2ce578757e None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:39.227747 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac7d5a-01dc-445e-a73a-ca2ce578757e None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:39.228217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac7d5a-01dc-445e-a73a-ca2ce578757e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:39.246830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 806/1624] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:39 2022] PUT /identity/v3/projects/6d5e228f10a440e784e7439dd6a1bfc8/users/98f76e47ee764cf094b372a5d8792f1f/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:39.247597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd90b40-d47a-45ee-9c03-ca53564acef5 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:39.267282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-d2ac7d5a-01dc-445e-a73a-ca2ce578757e None None] MFA Rules not processed for user `982b82e8ac424318a934f9149fb91706`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:39.269057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d2ac7d5a-01dc-445e-a73a-ca2ce578757e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:39.304817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:39.344681 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd90b40-d47a-45ee-9c03-ca53564acef5 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:39.351418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5b640-23f0-4ed5-9eba-8be940f0fa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:39.351986 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5b640-23f0-4ed5-9eba-8be940f0fa8d admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:39.352276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5b640-23f0-4ed5-9eba-8be940f0fa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:39.352554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5b640-23f0-4ed5-9eba-8be940f0fa8d admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:39.353282 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5b640-23f0-4ed5-9eba-8be940f0fa8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:39.355375 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5b640-23f0-4ed5-9eba-8be940f0fa8d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:39.392242 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 819/1625] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Sep 28 12:22:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:39.393206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac7d5a-01dc-445e-a73a-ca2ce578757e None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:39.399807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 807/1626] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:39 2022] POST /identity/v3/users => generated 291 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:39.401741 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5b640-23f0-4ed5-9eba-8be940f0fa8d admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:39.409413 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac7d5a-01dc-445e-a73a-ca2ce578757e None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:39.485761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930dce2e-b26e-4aa1-9575-05e02fc24752 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:39.485761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930dce2e-b26e-4aa1-9575-05e02fc24752 service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:39.486844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5b640-23f0-4ed5-9eba-8be940f0fa8d admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:39.488645 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930dce2e-b26e-4aa1-9575-05e02fc24752 service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:39.488645 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930dce2e-b26e-4aa1-9575-05e02fc24752 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:39.489350 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e69a1-21c9-4fde-bf5a-3be98811c0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:39.489350 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e69a1-21c9-4fde-bf5a-3be98811c0ee admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:39.489350 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e69a1-21c9-4fde-bf5a-3be98811c0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:39.489350 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e69a1-21c9-4fde-bf5a-3be98811c0ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:39.490039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e69a1-21c9-4fde-bf5a-3be98811c0ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:39.490039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e69a1-21c9-4fde-bf5a-3be98811c0ee admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:39.490039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 808/1627] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:39.490039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e69a1-21c9-4fde-bf5a-3be98811c0ee admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:39.515654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e69a1-21c9-4fde-bf5a-3be98811c0ee admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:39.525615 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47c461-c309-48f2-a3be-1cc45e66a7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:39.526202 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47c461-c309-48f2-a3be-1cc45e66a7d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:39.526489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47c461-c309-48f2-a3be-1cc45e66a7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:39.526810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47c461-c309-48f2-a3be-1cc45e66a7d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:39.527773 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47c461-c309-48f2-a3be-1cc45e66a7d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:39.529545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47c461-c309-48f2-a3be-1cc45e66a7d3 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:39.545560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 809/1628] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:39.546556 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47c461-c309-48f2-a3be-1cc45e66a7d3 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:39.567515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47c461-c309-48f2-a3be-1cc45e66a7d3 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:39.575492 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea9911-02f7-447f-ba30-7974910c859d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:39.576422 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea9911-02f7-447f-ba30-7974910c859d service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:39.576801 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea9911-02f7-447f-ba30-7974910c859d service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:39.577231 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea9911-02f7-447f-ba30-7974910c859d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:39.677878 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930dce2e-b26e-4aa1-9575-05e02fc24752 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:39.680765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930dce2e-b26e-4aa1-9575-05e02fc24752 service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:39.686529 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea9911-02f7-447f-ba30-7974910c859d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:39.688300 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea9911-02f7-447f-ba30-7974910c859d service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:39.697479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 820/1629] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:39.698939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930dce2e-b26e-4aa1-9575-05e02fc24752 service nova] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:39.715213 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930dce2e-b26e-4aa1-9575-05e02fc24752 service nova] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:39.725840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc94d3-9234-4fff-9924-e55b2da7b691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:39.728684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc94d3-9234-4fff-9924-e55b2da7b691 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:39.728963 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc94d3-9234-4fff-9924-e55b2da7b691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:39.730475 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc94d3-9234-4fff-9924-e55b2da7b691 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:39.731910 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc94d3-9234-4fff-9924-e55b2da7b691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:39.733789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 810/1630] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:39.734171 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc94d3-9234-4fff-9924-e55b2da7b691 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:39.738159 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea9911-02f7-447f-ba30-7974910c859d service nova] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:39.758401 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 821/1631] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:39 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:39.759308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc94d3-9234-4fff-9924-e55b2da7b691 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:39.783910 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea9911-02f7-447f-ba30-7974910c859d service nova] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:39.797384 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc94d3-9234-4fff-9924-e55b2da7b691 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:39.805946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0e208-fb20-4205-9d95-e3d6a5086d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:39.807417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0e208-fb20-4205-9d95-e3d6a5086d31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:39.811577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0e208-fb20-4205-9d95-e3d6a5086d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:39.813709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0e208-fb20-4205-9d95-e3d6a5086d31 admin admin] PATH_INFO: `/v3/projects/3dd1fbfaf05b4374ab1a3d380c1cb99e/users/6b5b956024104b4ca671511c57579e11/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:39.815807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7311d-27ba-4dc5-a6e5-6907721b1d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:39.816688 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7311d-27ba-4dc5-a6e5-6907721b1d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:39.816973 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7311d-27ba-4dc5-a6e5-6907721b1d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:39.820321 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7311d-27ba-4dc5-a6e5-6907721b1d1e admin admin] PATH_INFO: `/v3/projects/6d5e228f10a440e784e7439dd6a1bfc8/users/98f76e47ee764cf094b372a5d8792f1f/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:39.827036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7311d-27ba-4dc5-a6e5-6907721b1d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d5e228f10a440e784e7439dd6a1bfc8, user_id=98f76e47ee764cf094b372a5d8792f1f, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:39.837916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7311d-27ba-4dc5-a6e5-6907721b1d1e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:39.868189 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0e208-fb20-4205-9d95-e3d6a5086d31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dd1fbfaf05b4374ab1a3d380c1cb99e, user_id=6b5b956024104b4ca671511c57579e11, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:39.870991 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0e208-fb20-4205-9d95-e3d6a5086d31 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:39.885064 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 822/1632] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:39 2022] PUT /identity/v3/projects/6d5e228f10a440e784e7439dd6a1bfc8/users/98f76e47ee764cf094b372a5d8792f1f/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:39.887086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7311d-27ba-4dc5-a6e5-6907721b1d1e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:39.918126 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 811/1633] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:39 2022] PUT /identity/v3/projects/3dd1fbfaf05b4374ab1a3d380c1cb99e/users/6b5b956024104b4ca671511c57579e11/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:39.921195 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d7f20-41f8-4785-9e12-e9a873209fa9 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:39.921794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d7f20-41f8-4785-9e12-e9a873209fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:39.922298 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d7f20-41f8-4785-9e12-e9a873209fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:39.934966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:39.971631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-610d7f20-41f8-4785-9e12-e9a873209fa9 None None] MFA Rules not processed for user `98f76e47ee764cf094b372a5d8792f1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:39.977007 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-610d7f20-41f8-4785-9e12-e9a873209fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:40.064712 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7311d-27ba-4dc5-a6e5-6907721b1d1e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:40.223008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:40.234662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 812/1634] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:40.247803 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d7f20-41f8-4785-9e12-e9a873209fa9 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:40.274509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef57caa-596d-4bcc-a116-54e8d06ab032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:40.275295 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef57caa-596d-4bcc-a116-54e8d06ab032 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:40.276622 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef57caa-596d-4bcc-a116-54e8d06ab032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:40.279112 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef57caa-596d-4bcc-a116-54e8d06ab032 admin admin] PATH_INFO: `/v3/projects/a3d458a55eaf4ca3a3392b1c91dea027/users/03e1c3a512384cb196adcfa7f4fef20d/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:40.304351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef57caa-596d-4bcc-a116-54e8d06ab032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d458a55eaf4ca3a3392b1c91dea027, user_id=03e1c3a512384cb196adcfa7f4fef20d, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:40.307094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef57caa-596d-4bcc-a116-54e8d06ab032 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:40.331097 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 823/1635] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:39 2022] PUT /identity/v3/projects/a3d458a55eaf4ca3a3392b1c91dea027/users/03e1c3a512384cb196adcfa7f4fef20d/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:40.334869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef57caa-596d-4bcc-a116-54e8d06ab032 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:40.398104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d7f20-41f8-4785-9e12-e9a873209fa9 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:40.409664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef57caa-596d-4bcc-a116-54e8d06ab032 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:40.415373 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85620a35-2996-4771-8f2f-e585dd415c7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:40.416001 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85620a35-2996-4771-8f2f-e585dd415c7b service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:40.416283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85620a35-2996-4771-8f2f-e585dd415c7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:40.416672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85620a35-2996-4771-8f2f-e585dd415c7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:40.493389 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:40.505476 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44e124-8a44-4fcb-91db-66e74bb5e269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:40.506948 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85620a35-2996-4771-8f2f-e585dd415c7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:40.507907 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44e124-8a44-4fcb-91db-66e74bb5e269 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:40.508294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44e124-8a44-4fcb-91db-66e74bb5e269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:40.508669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44e124-8a44-4fcb-91db-66e74bb5e269 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:40.508803 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85620a35-2996-4771-8f2f-e585dd415c7b service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:40.509872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb44e124-8a44-4fcb-91db-66e74bb5e269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:40.511683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb44e124-8a44-4fcb-91db-66e74bb5e269 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:40.523470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 824/1636] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:40.524996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d466bc-5198-4389-98a8-f6f8c8f95f2f None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:40.525329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d466bc-5198-4389-98a8-f6f8c8f95f2f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:40.525611 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d466bc-5198-4389-98a8-f6f8c8f95f2f None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:40.527714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 825/1637] 10.222.0.60 () {54 vars in 912 bytes} [Wed Sep 28 12:22:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:22:40.528764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d466bc-5198-4389-98a8-f6f8c8f95f2f None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:40.547188 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d466bc-5198-4389-98a8-f6f8c8f95f2f None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:40.553915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846bb4b-c3fd-488e-b35b-ee2817d372f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:40.555140 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846bb4b-c3fd-488e-b35b-ee2817d372f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:40.555423 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846bb4b-c3fd-488e-b35b-ee2817d372f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:40.555884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846bb4b-c3fd-488e-b35b-ee2817d372f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:40.556539 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846bb4b-c3fd-488e-b35b-ee2817d372f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:40.558186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846bb4b-c3fd-488e-b35b-ee2817d372f6 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:40.581015 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 813/1638] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:40.582559 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9484ad6-78c6-447c-8f5f-4ade56450451 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:40.582954 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9484ad6-78c6-447c-8f5f-4ade56450451 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:40.583343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9484ad6-78c6-447c-8f5f-4ade56450451 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:40.591319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 814/1639] 10.222.0.60 () {54 vars in 912 bytes} [Wed Sep 28 12:22:40 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:22:40.594013 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a4fc9-474a-4602-a1a3-affb37cc9570 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:40.594417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a4fc9-474a-4602-a1a3-affb37cc9570 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:40.594796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a4fc9-474a-4602-a1a3-affb37cc9570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:40.599616 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 826/1640] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:40.602364 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846bb4b-c3fd-488e-b35b-ee2817d372f6 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:40.638483 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846bb4b-c3fd-488e-b35b-ee2817d372f6 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:40.645207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-614a4fc9-474a-4602-a1a3-affb37cc9570 None None] MFA Rules not processed for user `982b82e8ac424318a934f9149fb91706`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:40.646977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-614a4fc9-474a-4602-a1a3-affb37cc9570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:40.647234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5354c-b7a2-4174-8c7e-309f039ad6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:40.647926 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5354c-b7a2-4174-8c7e-309f039ad6cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:40.648207 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5354c-b7a2-4174-8c7e-309f039ad6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:40.648480 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5354c-b7a2-4174-8c7e-309f039ad6cb admin admin] PATH_INFO: `/v3/projects/3dd1fbfaf05b4374ab1a3d380c1cb99e/users/6b5b956024104b4ca671511c57579e11/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:40.654436 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5354c-b7a2-4174-8c7e-309f039ad6cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dd1fbfaf05b4374ab1a3d380c1cb99e, user_id=6b5b956024104b4ca671511c57579e11, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:40.659066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f5354c-b7a2-4174-8c7e-309f039ad6cb admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:40.686553 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 827/1641] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:40 2022] PUT /identity/v3/projects/3dd1fbfaf05b4374ab1a3d380c1cb99e/users/6b5b956024104b4ca671511c57579e11/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:40.688050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16910681-8b31-42f0-80a3-f4825886435b None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:40.688180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16910681-8b31-42f0-80a3-f4825886435b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:40.688455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16910681-8b31-42f0-80a3-f4825886435b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:40.723294 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-16910681-8b31-42f0-80a3-f4825886435b None None] MFA Rules not processed for user `982b82e8ac424318a934f9149fb91706`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:40.725206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-16910681-8b31-42f0-80a3-f4825886435b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:40.805956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 815/1642] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 12:22:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:40.807799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a4fc9-474a-4602-a1a3-affb37cc9570 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:40.824333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a4fc9-474a-4602-a1a3-affb37cc9570 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:40.855078 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 828/1643] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 12:22:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:40.856653 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb3087-cc7d-46f3-8a72-4d1ec698fc99 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:40.857125 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb3087-cc7d-46f3-8a72-4d1ec698fc99 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:40.858061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb3087-cc7d-46f3-8a72-4d1ec698fc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:40.882112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:40.893225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7dbb3087-cc7d-46f3-8a72-4d1ec698fc99 None None] MFA Rules not processed for user `6b5b956024104b4ca671511c57579e11`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:40.894605 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7dbb3087-cc7d-46f3-8a72-4d1ec698fc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:40.932602 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb0bc4-43c8-4de4-a422-7b54cc7ac966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:40.934239 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb0bc4-43c8-4de4-a422-7b54cc7ac966 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:40.937537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb0bc4-43c8-4de4-a422-7b54cc7ac966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:40.938108 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb0bc4-43c8-4de4-a422-7b54cc7ac966 admin admin] PATH_INFO: `/v3/projects/a3d458a55eaf4ca3a3392b1c91dea027/users/03e1c3a512384cb196adcfa7f4fef20d/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:40.940978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb0bc4-43c8-4de4-a422-7b54cc7ac966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d458a55eaf4ca3a3392b1c91dea027, user_id=03e1c3a512384cb196adcfa7f4fef20d, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:40.944396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb0bc4-43c8-4de4-a422-7b54cc7ac966 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:40.960953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 816/1644] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:40 2022] PUT /identity/v3/projects/a3d458a55eaf4ca3a3392b1c91dea027/users/03e1c3a512384cb196adcfa7f4fef20d/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:40.962245 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb0bc4-43c8-4de4-a422-7b54cc7ac966 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:41.001376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 829/1645] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:40 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:41.002897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb3087-cc7d-46f3-8a72-4d1ec698fc99 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:41.104234 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb0bc4-43c8-4de4-a422-7b54cc7ac966 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:41.108214 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc03699-ee6c-41a8-97c7-866d38ef266e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:41.108834 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc03699-ee6c-41a8-97c7-866d38ef266e service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:41.109370 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:41.109877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc03699-ee6c-41a8-97c7-866d38ef266e service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:41.110141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc03699-ee6c-41a8-97c7-866d38ef266e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:41.131968 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb3087-cc7d-46f3-8a72-4d1ec698fc99 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:41.136958 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a17a1-96c0-4bee-a256-4fa7b49bcde3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:41.138028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a17a1-96c0-4bee-a256-4fa7b49bcde3 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:41.138315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a17a1-96c0-4bee-a256-4fa7b49bcde3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:41.138591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a17a1-96c0-4bee-a256-4fa7b49bcde3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:41.214289 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc03699-ee6c-41a8-97c7-866d38ef266e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:41.215656 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc03699-ee6c-41a8-97c7-866d38ef266e service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:41.247577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 817/1646] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:41.249528 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520f38b-8e91-42d4-a026-3cd70d260a13 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:41.251101 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520f38b-8e91-42d4-a026-3cd70d260a13 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:41.252783 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520f38b-8e91-42d4-a026-3cd70d260a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:41.272930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a17a1-96c0-4bee-a256-4fa7b49bcde3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:41.274881 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a17a1-96c0-4bee-a256-4fa7b49bcde3 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:41.301980 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 830/1647] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:41.351317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-5520f38b-8e91-42d4-a026-3cd70d260a13 None None] MFA Rules not processed for user `03e1c3a512384cb196adcfa7f4fef20d`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:41.353273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5520f38b-8e91-42d4-a026-3cd70d260a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:41.555915 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 818/1648] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:42.135957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a17a1-96c0-4bee-a256-4fa7b49bcde3 service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:42.146153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a17a1-96c0-4bee-a256-4fa7b49bcde3 service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:42.163494 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e195962-798b-4f69-a12c-7ade0b803791 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:42.167582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e195962-798b-4f69-a12c-7ade0b803791 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:42.167582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e195962-798b-4f69-a12c-7ade0b803791 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:42.170479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e195962-798b-4f69-a12c-7ade0b803791 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:42.210915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e195962-798b-4f69-a12c-7ade0b803791 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:42.212730 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e195962-798b-4f69-a12c-7ade0b803791 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:42.258303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 831/1649] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:42.376449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520f38b-8e91-42d4-a026-3cd70d260a13 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:42.376449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520f38b-8e91-42d4-a026-3cd70d260a13 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:42.382596 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83ccd6-14ad-4dc9-896a-d7a72d02c726 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:42.382875 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83ccd6-14ad-4dc9-896a-d7a72d02c726 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:42.383161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83ccd6-14ad-4dc9-896a-d7a72d02c726 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:42.383438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83ccd6-14ad-4dc9-896a-d7a72d02c726 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:42.411735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:42.533054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e195962-798b-4f69-a12c-7ade0b803791 service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:42.544338 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e195962-798b-4f69-a12c-7ade0b803791 service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:42.560426 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f777971-efc5-407c-a7dd-57d02b801ec6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:42.560556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f777971-efc5-407c-a7dd-57d02b801ec6 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:42.560952 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f777971-efc5-407c-a7dd-57d02b801ec6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:42.561474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f777971-efc5-407c-a7dd-57d02b801ec6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:42.582435 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f777971-efc5-407c-a7dd-57d02b801ec6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:42.584053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f777971-efc5-407c-a7dd-57d02b801ec6 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:42.613658 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 832/1650] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:42.665092 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83ccd6-14ad-4dc9-896a-d7a72d02c726 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:42.666597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83ccd6-14ad-4dc9-896a-d7a72d02c726 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:42.712417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 819/1651] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:42 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:43.076043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f777971-efc5-407c-a7dd-57d02b801ec6 service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:43.276389 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f777971-efc5-407c-a7dd-57d02b801ec6 service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:43.303661 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c52f03-924b-4989-b87d-81748d1f772e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:43.303661 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c52f03-924b-4989-b87d-81748d1f772e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:43.303661 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c52f03-924b-4989-b87d-81748d1f772e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:43.303661 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c52f03-924b-4989-b87d-81748d1f772e admin admin] PATH_INFO: `/v3/users/0e4b2eb9978342ab80e698151fdcf049` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:43.304876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83ccd6-14ad-4dc9-896a-d7a72d02c726 service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:43.311051 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c52f03-924b-4989-b87d-81748d1f772e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e4b2eb9978342ab80e698151fdcf049)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:43.314900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c52f03-924b-4989-b87d-81748d1f772e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:43.323813 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83ccd6-14ad-4dc9-896a-d7a72d02c726 service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:43.342386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b3771-5943-477a-9cb2-19d9ee13708f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:43.346870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b3771-5943-477a-9cb2-19d9ee13708f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:43.348329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b3771-5943-477a-9cb2-19d9ee13708f admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:43.353110 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b3771-5943-477a-9cb2-19d9ee13708f admin admin] PATH_INFO: `/v3/users/d1d5082d04b843d0a31dda851bfff1d4` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:43.357943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b3771-5943-477a-9cb2-19d9ee13708f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d5082d04b843d0a31dda851bfff1d4)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:43.360739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: WARNING py.warnings [None req-fb9b3771-5943-477a-9cb2-19d9ee13708f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 12:22:43.360739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: warnings.warn(msg) Sep 28 12:22:43.360739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: Sep 28 12:22:43.368799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b3771-5943-477a-9cb2-19d9ee13708f admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:43.451545 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-43c52f03-924b-4989-b87d-81748d1f772e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e4b2eb9978342ab80e698151fdcf049'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:43.452245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-43c52f03-924b-4989-b87d-81748d1f772e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e4b2eb9978342ab80e698151fdcf049'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:43.468992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-43c52f03-924b-4989-b87d-81748d1f772e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e4b2eb9978342ab80e698151fdcf049'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:43.470483 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-43c52f03-924b-4989-b87d-81748d1f772e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e4b2eb9978342ab80e698151fdcf049'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:43.479865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:43.483714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-43c52f03-924b-4989-b87d-81748d1f772e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e4b2eb9978342ab80e698151fdcf049'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:43.541653 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 833/1652] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:22:43 2022] DELETE /identity/v3/users/0e4b2eb9978342ab80e698151fdcf049 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:43.546248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5898e30-7867-4035-88bf-73b31af986e3 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:43.548083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5898e30-7867-4035-88bf-73b31af986e3 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:43.548189 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5898e30-7867-4035-88bf-73b31af986e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:43.567847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-fb9b3771-5943-477a-9cb2-19d9ee13708f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d5082d04b843d0a31dda851bfff1d4'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:43.570289 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-fb9b3771-5943-477a-9cb2-19d9ee13708f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d5082d04b843d0a31dda851bfff1d4'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:43.608074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-b5898e30-7867-4035-88bf-73b31af986e3 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:43.616241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b5898e30-7867-4035-88bf-73b31af986e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:43.643595 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-fb9b3771-5943-477a-9cb2-19d9ee13708f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d5082d04b843d0a31dda851bfff1d4'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:43.682891 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-fb9b3771-5943-477a-9cb2-19d9ee13708f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d5082d04b843d0a31dda851bfff1d4'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:43.684961 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-fb9b3771-5943-477a-9cb2-19d9ee13708f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d5082d04b843d0a31dda851bfff1d4'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:43.738307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 820/1653] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:22:43 2022] DELETE /identity/v3/users/d1d5082d04b843d0a31dda851bfff1d4 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:43.746096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b52c6b-e15b-4282-a0e7-7e98b64cce6d None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:43.751937 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b52c6b-e15b-4282-a0e7-7e98b64cce6d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:43.752953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b52c6b-e15b-4282-a0e7-7e98b64cce6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:43.812530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 834/1654] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:43.832854 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-65b52c6b-e15b-4282-a0e7-7e98b64cce6d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:43.842512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-65b52c6b-e15b-4282-a0e7-7e98b64cce6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:43.980010 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5898e30-7867-4035-88bf-73b31af986e3 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:43.987445 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b5898e30-7867-4035-88bf-73b31af986e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:44.057302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 821/1655] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:44.134590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b52c6b-e15b-4282-a0e7-7e98b64cce6d None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:44.136276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-65b52c6b-e15b-4282-a0e7-7e98b64cce6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:44.172366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5898e30-7867-4035-88bf-73b31af986e3 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:44.183261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:44.387248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8813ca5-c031-4d76-bf72-9f2dfe263d20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:44.390021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8813ca5-c031-4d76-bf72-9f2dfe263d20 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:44.392793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8813ca5-c031-4d76-bf72-9f2dfe263d20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:44.393087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8813ca5-c031-4d76-bf72-9f2dfe263d20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:44.411183 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b52c6b-e15b-4282-a0e7-7e98b64cce6d None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:44.423397 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8813ca5-c031-4d76-bf72-9f2dfe263d20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:44.429362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1767fd-1ea6-4104-bef2-d8223d118243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:44.432781 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8813ca5-c031-4d76-bf72-9f2dfe263d20 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:44.434688 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1767fd-1ea6-4104-bef2-d8223d118243 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:44.435085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1767fd-1ea6-4104-bef2-d8223d118243 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:44.436879 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1767fd-1ea6-4104-bef2-d8223d118243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:44.459666 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 835/1656] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:22:44.487207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1767fd-1ea6-4104-bef2-d8223d118243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:44.489791 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1767fd-1ea6-4104-bef2-d8223d118243 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:44.532429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 822/1657] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:22:45.538539 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8813ca5-c031-4d76-bf72-9f2dfe263d20 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:45.540340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d8813ca5-c031-4d76-bf72-9f2dfe263d20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:45.570565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8813ca5-c031-4d76-bf72-9f2dfe263d20 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:45.582165 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:45.582822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:45.583101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:45.583375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8 admin admin] PATH_INFO: `/v3/projects/8476c581992e465fbd1f81526088b5eb` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:45.586250 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8476c581992e465fbd1f81526088b5eb)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:45.587290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: WARNING py.warnings [None req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 12:22:45.587290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: warnings.warn(msg) Sep 28 12:22:45.587290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: Sep 28 12:22:45.588555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:45.703529 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:45.742040 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8476c581992e465fbd1f81526088b5eb', 'request_id': 'req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:45.770428 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8476c581992e465fbd1f81526088b5eb', 'request_id': 'req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:45.813996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8476c581992e465fbd1f81526088b5eb', 'request_id': 'req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:45.855600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8 admin admin] The token cache is being invalidate because project 8476c581992e465fbd1f81526088b5eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:22:45.855905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:45.857957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:45.872003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 836/1658] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:22:45 2022] DELETE /identity/v3/projects/8476c581992e465fbd1f81526088b5eb => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:45.873584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1767fd-1ea6-4104-bef2-d8223d118243 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:45.903089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1767fd-1ea6-4104-bef2-d8223d118243 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:46.099317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30020c32-f8f3-412e-a9bb-62ffb4de83f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:46.100407 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30020c32-f8f3-412e-a9bb-62ffb4de83f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:46.100841 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30020c32-f8f3-412e-a9bb-62ffb4de83f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:46.101284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30020c32-f8f3-412e-a9bb-62ffb4de83f8 admin admin] PATH_INFO: `/v3/users/6b5b956024104b4ca671511c57579e11` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:46.103951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30020c32-f8f3-412e-a9bb-62ffb4de83f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b5b956024104b4ca671511c57579e11)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:46.106132 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30020c32-f8f3-412e-a9bb-62ffb4de83f8 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:46.224039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-30020c32-f8f3-412e-a9bb-62ffb4de83f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b5b956024104b4ca671511c57579e11'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:46.225855 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-30020c32-f8f3-412e-a9bb-62ffb4de83f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b5b956024104b4ca671511c57579e11'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:46.272090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-30020c32-f8f3-412e-a9bb-62ffb4de83f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b5b956024104b4ca671511c57579e11'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:46.293260 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:46.295420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:46.306969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-30020c32-f8f3-412e-a9bb-62ffb4de83f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b5b956024104b4ca671511c57579e11'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:46.307588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-30020c32-f8f3-412e-a9bb-62ffb4de83f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b5b956024104b4ca671511c57579e11'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:46.328211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb7150d-6e3d-4b41-a7c6-362c689cf4b8 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:46.363076 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db62b39-a3ea-4a66-9d08-6b898b7b0066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:46.364053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 823/1659] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:22:45 2022] DELETE /identity/v3/users/6b5b956024104b4ca671511c57579e11 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:46.373532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db62b39-a3ea-4a66-9d08-6b898b7b0066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:46.373817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db62b39-a3ea-4a66-9d08-6b898b7b0066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:46.374975 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db62b39-a3ea-4a66-9d08-6b898b7b0066 admin admin] PATH_INFO: `/v3/users/98290356ebed4635a170716391152f57` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:46.376545 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db62b39-a3ea-4a66-9d08-6b898b7b0066 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98290356ebed4635a170716391152f57)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:46.379187 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db62b39-a3ea-4a66-9d08-6b898b7b0066 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:46.497429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2db62b39-a3ea-4a66-9d08-6b898b7b0066 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98290356ebed4635a170716391152f57'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:46.498771 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2db62b39-a3ea-4a66-9d08-6b898b7b0066 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98290356ebed4635a170716391152f57'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:46.518462 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2db62b39-a3ea-4a66-9d08-6b898b7b0066 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98290356ebed4635a170716391152f57'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:46.520094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2db62b39-a3ea-4a66-9d08-6b898b7b0066 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98290356ebed4635a170716391152f57'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:46.538672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2db62b39-a3ea-4a66-9d08-6b898b7b0066 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98290356ebed4635a170716391152f57'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:46.596643 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 837/1660] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:22:46 2022] DELETE /identity/v3/users/98290356ebed4635a170716391152f57 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:22:46.601406 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc8766-bbbe-4d54-b6b5-bef006b726f6 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:46.602112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc8766-bbbe-4d54-b6b5-bef006b726f6 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:46.602444 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc8766-bbbe-4d54-b6b5-bef006b726f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:46.649647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-0ddc8766-bbbe-4d54-b6b5-bef006b726f6 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:46.659710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db62b39-a3ea-4a66-9d08-6b898b7b0066 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:46.661237 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0ddc8766-bbbe-4d54-b6b5-bef006b726f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:46.666057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2db62b39-a3ea-4a66-9d08-6b898b7b0066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:46.734194 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:46.827562 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 824/1661] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:46.919589 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db62b39-a3ea-4a66-9d08-6b898b7b0066 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:46.924459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc8766-bbbe-4d54-b6b5-bef006b726f6 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:46.926715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0ddc8766-bbbe-4d54-b6b5-bef006b726f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:46.942590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5643b8-84f1-41cb-b686-27de7f5d767b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:46.942743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5643b8-84f1-41cb-b686-27de7f5d767b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:46.944395 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5643b8-84f1-41cb-b686-27de7f5d767b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:46.944987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5643b8-84f1-41cb-b686-27de7f5d767b admin admin] PATH_INFO: `/v3/users/bc4f453670c846469b313199f6056839` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:46.947548 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5643b8-84f1-41cb-b686-27de7f5d767b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc4f453670c846469b313199f6056839)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:46.950813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5643b8-84f1-41cb-b686-27de7f5d767b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:47.060849 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:47.073480 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2b5643b8-84f1-41cb-b686-27de7f5d767b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc4f453670c846469b313199f6056839'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:47.074577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2b5643b8-84f1-41cb-b686-27de7f5d767b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc4f453670c846469b313199f6056839'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:47.090294 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2b5643b8-84f1-41cb-b686-27de7f5d767b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc4f453670c846469b313199f6056839'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:47.091294 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2b5643b8-84f1-41cb-b686-27de7f5d767b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc4f453670c846469b313199f6056839'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:47.101364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc8766-bbbe-4d54-b6b5-bef006b726f6 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:47.104697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2b5643b8-84f1-41cb-b686-27de7f5d767b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc4f453670c846469b313199f6056839'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:47.108744 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb77513-d0f1-4795-8bac-b27539397254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:47.109426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb77513-d0f1-4795-8bac-b27539397254 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:47.109706 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb77513-d0f1-4795-8bac-b27539397254 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:47.109969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb77513-d0f1-4795-8bac-b27539397254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:47.114171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7cb77513-d0f1-4795-8bac-b27539397254 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:47.146199 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 838/1662] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:22:46 2022] DELETE /identity/v3/users/bc4f453670c846469b313199f6056839 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:47.147430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5643b8-84f1-41cb-b686-27de7f5d767b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:47.148118 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb77513-d0f1-4795-8bac-b27539397254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:47.149626 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2b5643b8-84f1-41cb-b686-27de7f5d767b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:47.149759 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb77513-d0f1-4795-8bac-b27539397254 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:47.174196 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 825/1663] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:47.175662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec35ce-4291-4a63-9200-df504665bb33 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:47.176247 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec35ce-4291-4a63-9200-df504665bb33 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:47.176668 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec35ce-4291-4a63-9200-df504665bb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:47.238196 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-b1ec35ce-4291-4a63-9200-df504665bb33 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:47.239721 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b1ec35ce-4291-4a63-9200-df504665bb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:47.331566 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5643b8-84f1-41cb-b686-27de7f5d767b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:47.350971 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221c779-fe2d-4a1a-b9d3-7b560d6a0bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:47.356828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221c779-fe2d-4a1a-b9d3-7b560d6a0bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:47.357009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221c779-fe2d-4a1a-b9d3-7b560d6a0bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:47.360087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221c779-fe2d-4a1a-b9d3-7b560d6a0bfb admin admin] PATH_INFO: `/v3/users/7c5c1a11cd394a559d85a162cd82ae9c` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:47.363500 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221c779-fe2d-4a1a-b9d3-7b560d6a0bfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c5c1a11cd394a559d85a162cd82ae9c)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:47.370986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221c779-fe2d-4a1a-b9d3-7b560d6a0bfb admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:47.424974 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 826/1664] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:47.439595 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec35ce-4291-4a63-9200-df504665bb33 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:47.441177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b1ec35ce-4291-4a63-9200-df504665bb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:47.464471 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec35ce-4291-4a63-9200-df504665bb33 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:47.472709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f3e0e5-a1d1-4a6f-87ed-b02de09df057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:47.475424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3e0e5-a1d1-4a6f-87ed-b02de09df057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:47.475871 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3e0e5-a1d1-4a6f-87ed-b02de09df057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:47.476567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f3e0e5-a1d1-4a6f-87ed-b02de09df057 admin admin] PATH_INFO: `/v3/projects/f19c33b4bea04ee582a9424c2ddfdade` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:47.478314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f3e0e5-a1d1-4a6f-87ed-b02de09df057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f19c33b4bea04ee582a9424c2ddfdade)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:47.480920 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f3e0e5-a1d1-4a6f-87ed-b02de09df057 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:47.504660 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-7221c779-fe2d-4a1a-b9d3-7b560d6a0bfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c5c1a11cd394a559d85a162cd82ae9c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:47.506819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-7221c779-fe2d-4a1a-b9d3-7b560d6a0bfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c5c1a11cd394a559d85a162cd82ae9c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:47.526595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-7221c779-fe2d-4a1a-b9d3-7b560d6a0bfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c5c1a11cd394a559d85a162cd82ae9c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:47.527863 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-7221c779-fe2d-4a1a-b9d3-7b560d6a0bfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c5c1a11cd394a559d85a162cd82ae9c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:47.540094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-7221c779-fe2d-4a1a-b9d3-7b560d6a0bfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c5c1a11cd394a559d85a162cd82ae9c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:47.591602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 839/1665] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:22:47 2022] DELETE /identity/v3/users/7c5c1a11cd394a559d85a162cd82ae9c => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:47.593225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221c779-fe2d-4a1a-b9d3-7b560d6a0bfb admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:47.597570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7221c779-fe2d-4a1a-b9d3-7b560d6a0bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:47.600270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-40f3e0e5-a1d1-4a6f-87ed-b02de09df057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f19c33b4bea04ee582a9424c2ddfdade', 'request_id': 'req-40f3e0e5-a1d1-4a6f-87ed-b02de09df057'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:47.608782 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-40f3e0e5-a1d1-4a6f-87ed-b02de09df057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f19c33b4bea04ee582a9424c2ddfdade', 'request_id': 'req-40f3e0e5-a1d1-4a6f-87ed-b02de09df057'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:47.634014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-40f3e0e5-a1d1-4a6f-87ed-b02de09df057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f19c33b4bea04ee582a9424c2ddfdade', 'request_id': 'req-40f3e0e5-a1d1-4a6f-87ed-b02de09df057'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:47.653916 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-40f3e0e5-a1d1-4a6f-87ed-b02de09df057 admin admin] The token cache is being invalidate because project f19c33b4bea04ee582a9424c2ddfdade was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:22:47.654263 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-40f3e0e5-a1d1-4a6f-87ed-b02de09df057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:47.654981 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-40f3e0e5-a1d1-4a6f-87ed-b02de09df057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:47.658140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 827/1666] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:22:47 2022] DELETE /identity/v3/projects/f19c33b4bea04ee582a9424c2ddfdade => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:47.659694 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28757f30-0c03-4db8-8c06-6dbadce96537 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:47.660154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28757f30-0c03-4db8-8c06-6dbadce96537 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:47.660492 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28757f30-0c03-4db8-8c06-6dbadce96537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:47.674414 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:47.705397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-28757f30-0c03-4db8-8c06-6dbadce96537 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:47.709447 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:47.713516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-28757f30-0c03-4db8-8c06-6dbadce96537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:47.778440 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221c779-fe2d-4a1a-b9d3-7b560d6a0bfb admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:47.789398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b8661-3df5-4abd-b0e5-bc200d884181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:47.790043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b8661-3df5-4abd-b0e5-bc200d884181 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:47.790398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b8661-3df5-4abd-b0e5-bc200d884181 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:47.790677 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b8661-3df5-4abd-b0e5-bc200d884181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:47.797475 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-235b8661-3df5-4abd-b0e5-bc200d884181 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:47.887775 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 828/1667] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:47.892691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2bffa-30b3-40dd-84d9-ce2c0a8a0cd8 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:47.892987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2bffa-30b3-40dd-84d9-ce2c0a8a0cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:47.897121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2bffa-30b3-40dd-84d9-ce2c0a8a0cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:47.950700 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-18b2bffa-30b3-40dd-84d9-ce2c0a8a0cd8 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:47.951926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-18b2bffa-30b3-40dd-84d9-ce2c0a8a0cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:47.952044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b8661-3df5-4abd-b0e5-bc200d884181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:47.953694 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b8661-3df5-4abd-b0e5-bc200d884181 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:47.965843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 840/1668] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:47.966718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b8661-3df5-4abd-b0e5-bc200d884181 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:47.968086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-235b8661-3df5-4abd-b0e5-bc200d884181 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:47.970762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 829/1669] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:47.971998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b59036-30e0-4a78-a9fc-f63eadcf836d None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:47.972287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b59036-30e0-4a78-a9fc-f63eadcf836d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:47.972565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b59036-30e0-4a78-a9fc-f63eadcf836d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:47.988354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b8661-3df5-4abd-b0e5-bc200d884181 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:48.004056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864cbcc4-8e5e-4630-96b6-94e81b619ab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:48.005083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cbcc4-8e5e-4630-96b6-94e81b619ab7 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:48.006139 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cbcc4-8e5e-4630-96b6-94e81b619ab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:48.006745 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cbcc4-8e5e-4630-96b6-94e81b619ab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:48.028635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864cbcc4-8e5e-4630-96b6-94e81b619ab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:48.030268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864cbcc4-8e5e-4630-96b6-94e81b619ab7 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:48.047163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-f1b59036-30e0-4a78-a9fc-f63eadcf836d None None] MFA Rules not processed for user `4d7b7c68a699443aa5c0859094c265ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:48.048774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f1b59036-30e0-4a78-a9fc-f63eadcf836d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:48.059725 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 841/1670] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:48.063371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe120a40-aaf4-4a20-a90f-8686b1e1ed18 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:48.063847 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe120a40-aaf4-4a20-a90f-8686b1e1ed18 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:48.069149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe120a40-aaf4-4a20-a90f-8686b1e1ed18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:48.110150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-fe120a40-aaf4-4a20-a90f-8686b1e1ed18 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:48.111604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-fe120a40-aaf4-4a20-a90f-8686b1e1ed18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:48.132530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 842/1671] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:48.134053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe120a40-aaf4-4a20-a90f-8686b1e1ed18 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:48.143417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe120a40-aaf4-4a20-a90f-8686b1e1ed18 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:48.156831 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c5b40-d2a3-4882-9d43-e3cd442e6806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:48.157579 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c5b40-d2a3-4882-9d43-e3cd442e6806 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:48.157917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c5b40-d2a3-4882-9d43-e3cd442e6806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:48.158270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c5b40-d2a3-4882-9d43-e3cd442e6806 admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:48.158983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c5b40-d2a3-4882-9d43-e3cd442e6806 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:22:48.169459 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c5b40-d2a3-4882-9d43-e3cd442e6806 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:48.171251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c5b40-d2a3-4882-9d43-e3cd442e6806 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:48.189916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 843/1672] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:22:48.194212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c5b40-d2a3-4882-9d43-e3cd442e6806 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:48.199779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 830/1673] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:48.215359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b59036-30e0-4a78-a9fc-f63eadcf836d None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:48.216087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c5b40-d2a3-4882-9d43-e3cd442e6806 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:48.218048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f1b59036-30e0-4a78-a9fc-f63eadcf836d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:48.230397 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06831c4-6f38-4370-a97f-94a6c3749f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:48.231043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06831c4-6f38-4370-a97f-94a6c3749f3b admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:48.231323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06831c4-6f38-4370-a97f-94a6c3749f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:48.231724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06831c4-6f38-4370-a97f-94a6c3749f3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:48.232212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06831c4-6f38-4370-a97f-94a6c3749f3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:48.234199 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06831c4-6f38-4370-a97f-94a6c3749f3b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:48.296381 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 844/1674] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:48 2022] POST /identity/v3/projects => generated 403 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:48.300631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06831c4-6f38-4370-a97f-94a6c3749f3b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:48.413932 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b59036-30e0-4a78-a9fc-f63eadcf836d None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:48.438327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06831c4-6f38-4370-a97f-94a6c3749f3b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:48.451513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999bd67-1b60-407e-bc32-667a8d02605c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:48.452301 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999bd67-1b60-407e-bc32-667a8d02605c admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:48.452590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999bd67-1b60-407e-bc32-667a8d02605c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:48.452858 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999bd67-1b60-407e-bc32-667a8d02605c admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:48.454187 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999bd67-1b60-407e-bc32-667a8d02605c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:48.459629 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999bd67-1b60-407e-bc32-667a8d02605c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:48.502917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 845/1675] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:48 2022] POST /identity/v3/users => generated 302 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:48.503829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999bd67-1b60-407e-bc32-667a8d02605c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:48.508255 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:48.508952 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a999bd67-1b60-407e-bc32-667a8d02605c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:48.535142 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999bd67-1b60-407e-bc32-667a8d02605c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:48.542159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e87d49-37c6-4a2d-86a8-e78ef35b932d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:48.543494 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e87d49-37c6-4a2d-86a8-e78ef35b932d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:48.543918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e87d49-37c6-4a2d-86a8-e78ef35b932d admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:48.544332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e87d49-37c6-4a2d-86a8-e78ef35b932d admin admin] PATH_INFO: `/v3/projects/3dd1fbfaf05b4374ab1a3d380c1cb99e` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:48.545776 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e87d49-37c6-4a2d-86a8-e78ef35b932d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dd1fbfaf05b4374ab1a3d380c1cb99e)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:48.547876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e87d49-37c6-4a2d-86a8-e78ef35b932d admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:48.567882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:48.611778 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5956a4-1ab5-4cb9-a014-b491bbc198aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:48.612810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5956a4-1ab5-4cb9-a014-b491bbc198aa service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:48.613603 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5956a4-1ab5-4cb9-a014-b491bbc198aa service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:48.613603 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5956a4-1ab5-4cb9-a014-b491bbc198aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:48.683858 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-53e87d49-37c6-4a2d-86a8-e78ef35b932d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dd1fbfaf05b4374ab1a3d380c1cb99e', 'request_id': 'req-53e87d49-37c6-4a2d-86a8-e78ef35b932d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:48.702191 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-53e87d49-37c6-4a2d-86a8-e78ef35b932d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dd1fbfaf05b4374ab1a3d380c1cb99e', 'request_id': 'req-53e87d49-37c6-4a2d-86a8-e78ef35b932d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:48.713171 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-53e87d49-37c6-4a2d-86a8-e78ef35b932d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dd1fbfaf05b4374ab1a3d380c1cb99e', 'request_id': 'req-53e87d49-37c6-4a2d-86a8-e78ef35b932d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:48.731165 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-53e87d49-37c6-4a2d-86a8-e78ef35b932d admin admin] The token cache is being invalidate because project 3dd1fbfaf05b4374ab1a3d380c1cb99e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:22:48.731589 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-53e87d49-37c6-4a2d-86a8-e78ef35b932d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:48.732583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-53e87d49-37c6-4a2d-86a8-e78ef35b932d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:48.732864 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5956a4-1ab5-4cb9-a014-b491bbc198aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:48.734683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5956a4-1ab5-4cb9-a014-b491bbc198aa service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:48.735257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 846/1676] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:22:48 2022] DELETE /identity/v3/projects/3dd1fbfaf05b4374ab1a3d380c1cb99e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:48.736152 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e87d49-37c6-4a2d-86a8-e78ef35b932d admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:48.739885 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-53e87d49-37c6-4a2d-86a8-e78ef35b932d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:48.773633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 831/1677] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:22:48.776369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a7a32-7274-4def-a316-751dcee5adfe None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:48.776826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a7a32-7274-4def-a316-751dcee5adfe None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:48.777285 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a7a32-7274-4def-a316-751dcee5adfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:48.843620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-684a7a32-7274-4def-a316-751dcee5adfe None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:48.845023 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-684a7a32-7274-4def-a316-751dcee5adfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:48.853766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e87d49-37c6-4a2d-86a8-e78ef35b932d admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:48.875000 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafe492-a404-4fa8-8f28-4f93710e7681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:48.877640 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 832/1678] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:48.888068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59cb82-92fc-4954-904f-018806c53f70 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:48.888751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59cb82-92fc-4954-904f-018806c53f70 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:48.889287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59cb82-92fc-4954-904f-018806c53f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:48.890065 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafe492-a404-4fa8-8f28-4f93710e7681 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:48.890377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafe492-a404-4fa8-8f28-4f93710e7681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:48.890637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafe492-a404-4fa8-8f28-4f93710e7681 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:48.891321 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baafe492-a404-4fa8-8f28-4f93710e7681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:48.892948 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baafe492-a404-4fa8-8f28-4f93710e7681 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:48.927949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-2f59cb82-92fc-4954-904f-018806c53f70 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:48.930361 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 847/1679] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:48.931138 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2f59cb82-92fc-4954-904f-018806c53f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:48.931350 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafe492-a404-4fa8-8f28-4f93710e7681 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:48.955203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 833/1680] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:22:48.959970 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59cb82-92fc-4954-904f-018806c53f70 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:48.962097 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafe492-a404-4fa8-8f28-4f93710e7681 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:48.973364 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d9651-2ad3-480c-8ffe-e0e41b975529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:48.976464 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59cb82-92fc-4954-904f-018806c53f70 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:48.976890 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d9651-2ad3-480c-8ffe-e0e41b975529 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:48.977210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d9651-2ad3-480c-8ffe-e0e41b975529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:48.977533 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d9651-2ad3-480c-8ffe-e0e41b975529 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:48.978517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d9651-2ad3-480c-8ffe-e0e41b975529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:48.982587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d9651-2ad3-480c-8ffe-e0e41b975529 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:48.993929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dafcda-6f0a-4725-a793-78c216c87eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:48.994964 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dafcda-6f0a-4725-a793-78c216c87eec admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:48.995457 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dafcda-6f0a-4725-a793-78c216c87eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:48.996158 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dafcda-6f0a-4725-a793-78c216c87eec admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:48.997753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dafcda-6f0a-4725-a793-78c216c87eec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:22:48.998299 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dafcda-6f0a-4725-a793-78c216c87eec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:49.000158 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dafcda-6f0a-4725-a793-78c216c87eec admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:49.018555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 848/1681] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:49.020121 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d9651-2ad3-480c-8ffe-e0e41b975529 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:49.028656 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 834/1682] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:22:49.033328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dafcda-6f0a-4725-a793-78c216c87eec admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:49.040359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d9651-2ad3-480c-8ffe-e0e41b975529 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:49.046764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0981f5-c4f2-44c5-bd7f-fc953bd25c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:49.047799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0981f5-c4f2-44c5-bd7f-fc953bd25c0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:49.050008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0981f5-c4f2-44c5-bd7f-fc953bd25c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:49.050429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0981f5-c4f2-44c5-bd7f-fc953bd25c0b admin admin] PATH_INFO: `/v3/projects/20c8f2f67dce4d119916257372f7db75/users/ef81663b621e458899ad0eaa324a81fe/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:49.056313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dafcda-6f0a-4725-a793-78c216c87eec admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:49.067892 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa476c53-53fc-4043-ac32-9318b8890856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:49.068876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa476c53-53fc-4043-ac32-9318b8890856 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:49.069361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa476c53-53fc-4043-ac32-9318b8890856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:49.069807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa476c53-53fc-4043-ac32-9318b8890856 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:49.070659 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa476c53-53fc-4043-ac32-9318b8890856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:49.073052 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa476c53-53fc-4043-ac32-9318b8890856 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:49.074512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0981f5-c4f2-44c5-bd7f-fc953bd25c0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c8f2f67dce4d119916257372f7db75, user_id=ef81663b621e458899ad0eaa324a81fe, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:49.077257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0981f5-c4f2-44c5-bd7f-fc953bd25c0b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:49.109161 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 849/1683] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:49 2022] PUT /identity/v3/projects/20c8f2f67dce4d119916257372f7db75/users/ef81663b621e458899ad0eaa324a81fe/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:49.111479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d40ede-31bf-4f45-80d1-23d55491cc6a None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:49.111884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d40ede-31bf-4f45-80d1-23d55491cc6a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:49.112233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d40ede-31bf-4f45-80d1-23d55491cc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:49.147647 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-b5d40ede-31bf-4f45-80d1-23d55491cc6a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:49.153295 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b5d40ede-31bf-4f45-80d1-23d55491cc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:49.157321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 835/1684] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:49 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:22:49.161401 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa476c53-53fc-4043-ac32-9318b8890856 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:49.300548 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:49.310074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa476c53-53fc-4043-ac32-9318b8890856 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:49.319873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 850/1685] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:49.322661 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d40ede-31bf-4f45-80d1-23d55491cc6a None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:49.327438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c8cb2-9253-48ff-b77e-85d017975821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:49.328157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b5d40ede-31bf-4f45-80d1-23d55491cc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:49.328653 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c8cb2-9253-48ff-b77e-85d017975821 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:49.329810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c8cb2-9253-48ff-b77e-85d017975821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:49.330587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c8cb2-9253-48ff-b77e-85d017975821 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:49.331658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c8cb2-9253-48ff-b77e-85d017975821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:49.336384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c8cb2-9253-48ff-b77e-85d017975821 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:49.367132 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d40ede-31bf-4f45-80d1-23d55491cc6a None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:49.384874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 836/1686] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:49 2022] POST /identity/v3/users => generated 287 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:22:49.385056 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c8cb2-9253-48ff-b77e-85d017975821 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:49.389181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d5475-adb7-4cd1-90a3-896caea3bcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:49.389991 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d5475-adb7-4cd1-90a3-896caea3bcea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:49.390106 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d5475-adb7-4cd1-90a3-896caea3bcea admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:49.390407 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d5475-adb7-4cd1-90a3-896caea3bcea admin admin] PATH_INFO: `/v3/projects/b9be06233afe44b2a7bc6da9e971056e` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:49.391498 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d5475-adb7-4cd1-90a3-896caea3bcea admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9be06233afe44b2a7bc6da9e971056e)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:49.393388 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d5475-adb7-4cd1-90a3-896caea3bcea admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:49.406440 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-bd8c8cb2-9253-48ff-b77e-85d017975821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:49.417496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:49.529726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c8cb2-9253-48ff-b77e-85d017975821 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:49.534180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f66d5475-adb7-4cd1-90a3-896caea3bcea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9be06233afe44b2a7bc6da9e971056e', 'request_id': 'req-f66d5475-adb7-4cd1-90a3-896caea3bcea'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:49.542588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f66d5475-adb7-4cd1-90a3-896caea3bcea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9be06233afe44b2a7bc6da9e971056e', 'request_id': 'req-f66d5475-adb7-4cd1-90a3-896caea3bcea'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:49.562329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f66d5475-adb7-4cd1-90a3-896caea3bcea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9be06233afe44b2a7bc6da9e971056e', 'request_id': 'req-f66d5475-adb7-4cd1-90a3-896caea3bcea'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:49.583843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f66d5475-adb7-4cd1-90a3-896caea3bcea admin admin] The token cache is being invalidate because project b9be06233afe44b2a7bc6da9e971056e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:22:49.584293 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f66d5475-adb7-4cd1-90a3-896caea3bcea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:49.586233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f66d5475-adb7-4cd1-90a3-896caea3bcea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:49.590277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 851/1687] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:22:49 2022] DELETE /identity/v3/projects/b9be06233afe44b2a7bc6da9e971056e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:49.591982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d5475-adb7-4cd1-90a3-896caea3bcea admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:49.596276 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-f66d5475-adb7-4cd1-90a3-896caea3bcea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:49.658584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1adf02-9736-4307-adbc-f53c8c1507a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:49.661418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1adf02-9736-4307-adbc-f53c8c1507a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:49.663159 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1adf02-9736-4307-adbc-f53c8c1507a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:49.663455 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1adf02-9736-4307-adbc-f53c8c1507a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:49.670627 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-9a1adf02-9736-4307-adbc-f53c8c1507a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:49.752494 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d5475-adb7-4cd1-90a3-896caea3bcea admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:49.766820 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa350d13-2f67-494d-b33d-c1391f4ffa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:49.767361 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa350d13-2f67-494d-b33d-c1391f4ffa13 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:49.767638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa350d13-2f67-494d-b33d-c1391f4ffa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:49.768207 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa350d13-2f67-494d-b33d-c1391f4ffa13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:49.769306 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa350d13-2f67-494d-b33d-c1391f4ffa13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:49.772910 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa350d13-2f67-494d-b33d-c1391f4ffa13 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:49.798713 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 852/1688] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:49.798713 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa350d13-2f67-494d-b33d-c1391f4ffa13 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:49.802447 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-aa350d13-2f67-494d-b33d-c1391f4ffa13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:49.825354 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1adf02-9736-4307-adbc-f53c8c1507a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:49.826831 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1adf02-9736-4307-adbc-f53c8c1507a5 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:49.831407 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa350d13-2f67-494d-b33d-c1391f4ffa13 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:49.842058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:49.843710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:49.844050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:49.844268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00 admin admin] PATH_INFO: `/v3/projects/9cdf002f6eec42199d3619b26dcb7384` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:49.847380 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cdf002f6eec42199d3619b26dcb7384)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:49.848874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 837/1689] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:49.851357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:49.851930 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1adf02-9736-4307-adbc-f53c8c1507a5 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:49.854512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-9a1adf02-9736-4307-adbc-f53c8c1507a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:49.885288 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1adf02-9736-4307-adbc-f53c8c1507a5 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:49.899254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858efb7-5cfb-42fe-8a79-0ae984fb8a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:49.899877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858efb7-5cfb-42fe-8a79-0ae984fb8a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:49.900162 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858efb7-5cfb-42fe-8a79-0ae984fb8a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:49.900438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858efb7-5cfb-42fe-8a79-0ae984fb8a09 admin admin] PATH_INFO: `/v3/users/98f76e47ee764cf094b372a5d8792f1f` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:49.902650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858efb7-5cfb-42fe-8a79-0ae984fb8a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98f76e47ee764cf094b372a5d8792f1f)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:49.904702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858efb7-5cfb-42fe-8a79-0ae984fb8a09 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:49.956906 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cdf002f6eec42199d3619b26dcb7384', 'request_id': 'req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:49.970769 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cdf002f6eec42199d3619b26dcb7384', 'request_id': 'req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:49.989294 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cdf002f6eec42199d3619b26dcb7384', 'request_id': 'req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.030204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-9858efb7-5cfb-42fe-8a79-0ae984fb8a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98f76e47ee764cf094b372a5d8792f1f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.031771 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00 admin admin] The token cache is being invalidate because project 9cdf002f6eec42199d3619b26dcb7384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:22:50.032065 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.032610 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-9858efb7-5cfb-42fe-8a79-0ae984fb8a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98f76e47ee764cf094b372a5d8792f1f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.038703 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.043415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 853/1690] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:22:49 2022] DELETE /identity/v3/projects/9cdf002f6eec42199d3619b26dcb7384 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:50.046548 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:50.055233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:50.072283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:50.087846 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-9858efb7-5cfb-42fe-8a79-0ae984fb8a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98f76e47ee764cf094b372a5d8792f1f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.090726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:50.110541 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-9858efb7-5cfb-42fe-8a79-0ae984fb8a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98f76e47ee764cf094b372a5d8792f1f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.111392 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-9858efb7-5cfb-42fe-8a79-0ae984fb8a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98f76e47ee764cf094b372a5d8792f1f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.136993 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 838/1691] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:22:49 2022] DELETE /identity/v3/users/98f76e47ee764cf094b372a5d8792f1f => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:50.137978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858efb7-5cfb-42fe-8a79-0ae984fb8a09 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:50.143801 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-9858efb7-5cfb-42fe-8a79-0ae984fb8a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:50.237603 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8a2fc3-05ae-4fd8-ab32-5bab9e35cb00 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:50.306448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858efb7-5cfb-42fe-8a79-0ae984fb8a09 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:50.337323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a482ee-cc71-4ce7-8423-c1517d4e44f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:50.339267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a482ee-cc71-4ce7-8423-c1517d4e44f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:50.339897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a482ee-cc71-4ce7-8423-c1517d4e44f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:50.340390 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a482ee-cc71-4ce7-8423-c1517d4e44f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:50.341322 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a482ee-cc71-4ce7-8423-c1517d4e44f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:50.343349 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a482ee-cc71-4ce7-8423-c1517d4e44f4 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:50.350323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26de30e-f551-41fc-b4e0-12e4f454390c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:50.351522 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26de30e-f551-41fc-b4e0-12e4f454390c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:50.351810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26de30e-f551-41fc-b4e0-12e4f454390c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:50.352145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26de30e-f551-41fc-b4e0-12e4f454390c admin admin] PATH_INFO: `/v3/projects/bb55541c8a31486494dac11aa5c84e37` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:50.353656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26de30e-f551-41fc-b4e0-12e4f454390c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb55541c8a31486494dac11aa5c84e37)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:50.357111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26de30e-f551-41fc-b4e0-12e4f454390c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:50.361294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 839/1692] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:50.363118 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a482ee-cc71-4ce7-8423-c1517d4e44f4 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:50.366348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-50a482ee-cc71-4ce7-8423-c1517d4e44f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:50.386430 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a482ee-cc71-4ce7-8423-c1517d4e44f4 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:50.392550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de20bdda-e470-4b7b-a47a-3a0f62123b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:50.394292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20bdda-e470-4b7b-a47a-3a0f62123b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:50.394837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20bdda-e470-4b7b-a47a-3a0f62123b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:50.395845 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20bdda-e470-4b7b-a47a-3a0f62123b3b admin admin] PATH_INFO: `/v3/users/03e1c3a512384cb196adcfa7f4fef20d` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:50.398051 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de20bdda-e470-4b7b-a47a-3a0f62123b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=03e1c3a512384cb196adcfa7f4fef20d)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:50.400989 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de20bdda-e470-4b7b-a47a-3a0f62123b3b admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:50.452746 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b26de30e-f551-41fc-b4e0-12e4f454390c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb55541c8a31486494dac11aa5c84e37', 'request_id': 'req-b26de30e-f551-41fc-b4e0-12e4f454390c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.464112 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b26de30e-f551-41fc-b4e0-12e4f454390c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb55541c8a31486494dac11aa5c84e37', 'request_id': 'req-b26de30e-f551-41fc-b4e0-12e4f454390c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.484960 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b26de30e-f551-41fc-b4e0-12e4f454390c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb55541c8a31486494dac11aa5c84e37', 'request_id': 'req-b26de30e-f551-41fc-b4e0-12e4f454390c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.496298 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-de20bdda-e470-4b7b-a47a-3a0f62123b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03e1c3a512384cb196adcfa7f4fef20d'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.497056 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-de20bdda-e470-4b7b-a47a-3a0f62123b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03e1c3a512384cb196adcfa7f4fef20d'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.517516 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b26de30e-f551-41fc-b4e0-12e4f454390c admin admin] The token cache is being invalidate because project bb55541c8a31486494dac11aa5c84e37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:22:50.518144 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b26de30e-f551-41fc-b4e0-12e4f454390c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.520169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b26de30e-f551-41fc-b4e0-12e4f454390c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.524305 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-de20bdda-e470-4b7b-a47a-3a0f62123b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03e1c3a512384cb196adcfa7f4fef20d'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.525565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 854/1693] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:22:50 2022] DELETE /identity/v3/projects/bb55541c8a31486494dac11aa5c84e37 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:50.527069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26de30e-f551-41fc-b4e0-12e4f454390c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:50.532048 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b26de30e-f551-41fc-b4e0-12e4f454390c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:50.535897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-de20bdda-e470-4b7b-a47a-3a0f62123b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03e1c3a512384cb196adcfa7f4fef20d'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.537421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-de20bdda-e470-4b7b-a47a-3a0f62123b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03e1c3a512384cb196adcfa7f4fef20d'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:50.582882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 840/1694] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:22:50 2022] DELETE /identity/v3/users/03e1c3a512384cb196adcfa7f4fef20d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:50.587262 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c0780-8206-4b02-b5ab-21ab07869766 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:50.587522 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c0780-8206-4b02-b5ab-21ab07869766 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:50.587828 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c0780-8206-4b02-b5ab-21ab07869766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:50.628432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-5a6c0780-8206-4b02-b5ab-21ab07869766 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:50.630729 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5a6c0780-8206-4b02-b5ab-21ab07869766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:50.638196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:50.691989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26de30e-f551-41fc-b4e0-12e4f454390c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:50.737028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:50.803840 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 841/1695] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:50.811852 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f8b40-450c-492c-adf2-fa35a792f62d None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:50.812473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f8b40-450c-492c-adf2-fa35a792f62d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:50.812950 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f8b40-450c-492c-adf2-fa35a792f62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:50.824988 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1601010-6d9c-4e46-bfea-c22d1e2146d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:50.825901 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1601010-6d9c-4e46-bfea-c22d1e2146d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:50.826257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1601010-6d9c-4e46-bfea-c22d1e2146d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:50.829201 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1601010-6d9c-4e46-bfea-c22d1e2146d0 admin admin] PATH_INFO: `/v3/projects/fbfa9c57f947478b9eda820cf3f89303/users/4cda5c4a66264ee0816618dccb0011ec/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:50.865627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1601010-6d9c-4e46-bfea-c22d1e2146d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbfa9c57f947478b9eda820cf3f89303, user_id=4cda5c4a66264ee0816618dccb0011ec, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:50.869297 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1601010-6d9c-4e46-bfea-c22d1e2146d0 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:50.873440 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-0a3f8b40-450c-492c-adf2-fa35a792f62d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:50.876170 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0a3f8b40-450c-492c-adf2-fa35a792f62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:50.895304 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 855/1696] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:50 2022] PUT /identity/v3/projects/fbfa9c57f947478b9eda820cf3f89303/users/4cda5c4a66264ee0816618dccb0011ec/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:50.903933 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 842/1697] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:22:50.909473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cf110-f263-484a-b6fc-0471dce381e1 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:50.909839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cf110-f263-484a-b6fc-0471dce381e1 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:50.910289 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9cf110-f263-484a-b6fc-0471dce381e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:50.913006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f8b40-450c-492c-adf2-fa35a792f62d None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:50.948156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f8b40-450c-492c-adf2-fa35a792f62d None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:50.977662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-8c9cf110-f263-484a-b6fc-0471dce381e1 None None] MFA Rules not processed for user `4cda5c4a66264ee0816618dccb0011ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:50.979094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-8c9cf110-f263-484a-b6fc-0471dce381e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:51.092185 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7da15-5607-49b9-9e89-11540e39329f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:51.092817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7da15-5607-49b9-9e89-11540e39329f admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:51.095382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7da15-5607-49b9-9e89-11540e39329f admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:51.095382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7da15-5607-49b9-9e89-11540e39329f admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:51.095382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7da15-5607-49b9-9e89-11540e39329f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:22:51.095382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7da15-5607-49b9-9e89-11540e39329f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:51.096122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7da15-5607-49b9-9e89-11540e39329f admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:51.123424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 843/1698] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:22:51.124387 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7da15-5607-49b9-9e89-11540e39329f admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:51.148510 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7da15-5607-49b9-9e89-11540e39329f admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:51.169273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66db9d1-ad77-479d-a2fe-b4df2e7f1895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:51.169273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66db9d1-ad77-479d-a2fe-b4df2e7f1895 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:51.169273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66db9d1-ad77-479d-a2fe-b4df2e7f1895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:51.169273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66db9d1-ad77-479d-a2fe-b4df2e7f1895 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:51.170475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66db9d1-ad77-479d-a2fe-b4df2e7f1895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:51.170980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66db9d1-ad77-479d-a2fe-b4df2e7f1895 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:51.173098 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 856/1699] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:51.177352 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cf110-f263-484a-b6fc-0471dce381e1 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:51.203469 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9cf110-f263-484a-b6fc-0471dce381e1 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:51.213495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba4286-5a6a-42a1-98f2-3e43fbb0c8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:51.214466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba4286-5a6a-42a1-98f2-3e43fbb0c8e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:51.214761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba4286-5a6a-42a1-98f2-3e43fbb0c8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:51.215038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba4286-5a6a-42a1-98f2-3e43fbb0c8e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:51.215678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba4286-5a6a-42a1-98f2-3e43fbb0c8e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:51.217619 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba4286-5a6a-42a1-98f2-3e43fbb0c8e6 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:51.247643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 844/1700] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:51 2022] POST /identity/v3/projects => generated 367 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:51.254546 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66db9d1-ad77-479d-a2fe-b4df2e7f1895 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:51.287923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 857/1701] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:51 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:22:51.289017 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba4286-5a6a-42a1-98f2-3e43fbb0c8e6 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:51.430896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:51.440789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66db9d1-ad77-479d-a2fe-b4df2e7f1895 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:51.481777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba4286-5a6a-42a1-98f2-3e43fbb0c8e6 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:51.501257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fdc96-63a6-407f-bbf8-21162b83e387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:51.502144 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fdc96-63a6-407f-bbf8-21162b83e387 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:51.503075 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fdc96-63a6-407f-bbf8-21162b83e387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:51.503669 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fdc96-63a6-407f-bbf8-21162b83e387 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:51.504484 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fdc96-63a6-407f-bbf8-21162b83e387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:51.507929 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fdc96-63a6-407f-bbf8-21162b83e387 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:51.580242 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 858/1702] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:51 2022] POST /identity/v3/users => generated 292 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:51.581470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fdc96-63a6-407f-bbf8-21162b83e387 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:51.600234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fdc96-63a6-407f-bbf8-21162b83e387 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:51.610531 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849ff62-e30e-42a2-90c9-61008f314f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:51.612437 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d069a53-a020-4ad0-8da6-37b07512790b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:51.612818 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d069a53-a020-4ad0-8da6-37b07512790b admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:51.613105 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d069a53-a020-4ad0-8da6-37b07512790b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:51.613342 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849ff62-e30e-42a2-90c9-61008f314f83 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:51.613547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d069a53-a020-4ad0-8da6-37b07512790b admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:51.614297 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d069a53-a020-4ad0-8da6-37b07512790b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:51.617044 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849ff62-e30e-42a2-90c9-61008f314f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:51.617956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849ff62-e30e-42a2-90c9-61008f314f83 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:51.618399 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d069a53-a020-4ad0-8da6-37b07512790b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:51.619243 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849ff62-e30e-42a2-90c9-61008f314f83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:51.626204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849ff62-e30e-42a2-90c9-61008f314f83 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:51.643769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:51.669053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 859/1703] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:51.676129 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 845/1704] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:51 2022] POST /identity/v3/users => generated 284 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:51.682050 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849ff62-e30e-42a2-90c9-61008f314f83 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:51.685877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d069a53-a020-4ad0-8da6-37b07512790b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:51.702050 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849ff62-e30e-42a2-90c9-61008f314f83 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:51.706030 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d069a53-a020-4ad0-8da6-37b07512790b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:51.713694 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc75b7-18f9-42bf-80a0-62525ba608ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:51.715301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b22ce3-a934-443e-9404-29e8829d58d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:51.715703 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc75b7-18f9-42bf-80a0-62525ba608ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:51.715938 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b22ce3-a934-443e-9404-29e8829d58d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:51.716217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b22ce3-a934-443e-9404-29e8829d58d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:51.716490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b22ce3-a934-443e-9404-29e8829d58d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:51.717148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b22ce3-a934-443e-9404-29e8829d58d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:51.717672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc75b7-18f9-42bf-80a0-62525ba608ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:51.719300 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc75b7-18f9-42bf-80a0-62525ba608ae admin admin] PATH_INFO: `/v3/projects/cd7220eee3ef426ab997351674bfd6f2/users/64e1d753f61e4b97b72ac163078b6a6f/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:51.723500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b22ce3-a934-443e-9404-29e8829d58d7 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:51.752505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 846/1705] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:51.757404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b22ce3-a934-443e-9404-29e8829d58d7 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:51.765630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc75b7-18f9-42bf-80a0-62525ba608ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7220eee3ef426ab997351674bfd6f2, user_id=64e1d753f61e4b97b72ac163078b6a6f, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:51.776281 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc75b7-18f9-42bf-80a0-62525ba608ae admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:51.780501 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b22ce3-a934-443e-9404-29e8829d58d7 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:51.793993 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58a0de-3c34-4402-b501-d681317d1664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:51.794718 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58a0de-3c34-4402-b501-d681317d1664 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:51.794938 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58a0de-3c34-4402-b501-d681317d1664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:51.795208 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58a0de-3c34-4402-b501-d681317d1664 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:51.795913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58a0de-3c34-4402-b501-d681317d1664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:51.805071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58a0de-3c34-4402-b501-d681317d1664 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:51.811485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 860/1706] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:51 2022] PUT /identity/v3/projects/cd7220eee3ef426ab997351674bfd6f2/users/64e1d753f61e4b97b72ac163078b6a6f/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:51.812535 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc75b7-18f9-42bf-80a0-62525ba608ae admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:51.836259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 847/1707] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:51.840888 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58a0de-3c34-4402-b501-d681317d1664 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:51.866316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58a0de-3c34-4402-b501-d681317d1664 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:52.086065 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf7611-9904-440e-9a3a-9ce02331a5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:52.088810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf7611-9904-440e-9a3a-9ce02331a5dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:52.089170 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf7611-9904-440e-9a3a-9ce02331a5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:52.089547 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf7611-9904-440e-9a3a-9ce02331a5dc admin admin] PATH_INFO: `/v3/projects/aa5d37fd5ed34d46ab7e58ee52b127a4/users/f52bb23291ac4b5db5adee5788c06704/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:52.126273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc75b7-18f9-42bf-80a0-62525ba608ae admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:52.135012 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf7611-9904-440e-9a3a-9ce02331a5dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa5d37fd5ed34d46ab7e58ee52b127a4, user_id=f52bb23291ac4b5db5adee5788c06704, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:52.142958 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf7611-9904-440e-9a3a-9ce02331a5dc admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:52.147297 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556c2e1-61d6-4e27-a393-a4bc4d9002c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:52.148460 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556c2e1-61d6-4e27-a393-a4bc4d9002c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:52.149096 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556c2e1-61d6-4e27-a393-a4bc4d9002c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:52.149694 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556c2e1-61d6-4e27-a393-a4bc4d9002c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:52.150563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556c2e1-61d6-4e27-a393-a4bc4d9002c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:52.152893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556c2e1-61d6-4e27-a393-a4bc4d9002c6 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:52.200583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 848/1708] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:51 2022] PUT /identity/v3/projects/aa5d37fd5ed34d46ab7e58ee52b127a4/users/f52bb23291ac4b5db5adee5788c06704/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:52.211215 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 861/1709] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:52.211895 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf7611-9904-440e-9a3a-9ce02331a5dc admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:52.211895 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-fbdf7611-9904-440e-9a3a-9ce02331a5dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:52.212521 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556c2e1-61d6-4e27-a393-a4bc4d9002c6 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:52.241101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556c2e1-61d6-4e27-a393-a4bc4d9002c6 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:52.390249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf7611-9904-440e-9a3a-9ce02331a5dc admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:52.392653 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88579a8c-45d9-4022-bca9-ed623c9b3302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:52.393709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88579a8c-45d9-4022-bca9-ed623c9b3302 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:52.394159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88579a8c-45d9-4022-bca9-ed623c9b3302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:52.394589 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88579a8c-45d9-4022-bca9-ed623c9b3302 admin admin] PATH_INFO: `/v3/projects/cd7220eee3ef426ab997351674bfd6f2/users/64e1d753f61e4b97b72ac163078b6a6f/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:52.397289 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88579a8c-45d9-4022-bca9-ed623c9b3302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7220eee3ef426ab997351674bfd6f2, user_id=64e1d753f61e4b97b72ac163078b6a6f, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:52.398131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:52.399046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:52.399406 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:52.399752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d admin admin] PATH_INFO: `/v3/projects/a3d458a55eaf4ca3a3392b1c91dea027` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:52.400870 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88579a8c-45d9-4022-bca9-ed623c9b3302 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:52.401284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3d458a55eaf4ca3a3392b1c91dea027)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:52.403322 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:52.436679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 862/1710] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:52 2022] PUT /identity/v3/projects/cd7220eee3ef426ab997351674bfd6f2/users/64e1d753f61e4b97b72ac163078b6a6f/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:52.438449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896a8d6-2e40-40e2-9de3-d0c3d6d28caa None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:52.438882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896a8d6-2e40-40e2-9de3-d0c3d6d28caa None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:52.439303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896a8d6-2e40-40e2-9de3-d0c3d6d28caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:52.476244 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-5896a8d6-2e40-40e2-9de3-d0c3d6d28caa None None] MFA Rules not processed for user `f52bb23291ac4b5db5adee5788c06704`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:52.478074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5896a8d6-2e40-40e2-9de3-d0c3d6d28caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:52.500844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:52.505171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3d458a55eaf4ca3a3392b1c91dea027', 'request_id': 'req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:52.526785 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3d458a55eaf4ca3a3392b1c91dea027', 'request_id': 'req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:52.534152 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:52.556569 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3d458a55eaf4ca3a3392b1c91dea027', 'request_id': 'req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:52.589615 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d admin admin] The token cache is being invalidate because project a3d458a55eaf4ca3a3392b1c91dea027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:22:52.589931 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:52.592304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:52.600425 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 849/1711] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:22:52 2022] DELETE /identity/v3/projects/a3d458a55eaf4ca3a3392b1c91dea027 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:52.601706 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:52.603295 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:52.656518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 863/1712] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:52.663186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bf24c-dab1-4e6c-8a31-33f99bcff0ec None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:52.664139 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bf24c-dab1-4e6c-8a31-33f99bcff0ec None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:52.664139 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6bf24c-dab1-4e6c-8a31-33f99bcff0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:52.729931 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-1f6bf24c-dab1-4e6c-8a31-33f99bcff0ec None None] MFA Rules not processed for user `64e1d753f61e4b97b72ac163078b6a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:52.731436 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-1f6bf24c-dab1-4e6c-8a31-33f99bcff0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:52.744230 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf561d8-1558-4ba9-ba0f-d382e3ab4e7d admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:52.758381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:52.760756 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:52.761060 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:52.761364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6 admin admin] PATH_INFO: `/v3/projects/6d5e228f10a440e784e7439dd6a1bfc8` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:52.766888 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d5e228f10a440e784e7439dd6a1bfc8)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:52.769059 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:52.885304 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 864/1713] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:52.886584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfd1d9-a46d-422a-a30e-c55ac7fb0895 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:52.886868 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfd1d9-a46d-422a-a30e-c55ac7fb0895 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:52.887138 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfd1d9-a46d-422a-a30e-c55ac7fb0895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:52.890462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d5e228f10a440e784e7439dd6a1bfc8', 'request_id': 'req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:52.900575 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d5e228f10a440e784e7439dd6a1bfc8', 'request_id': 'req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:52.916332 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d5e228f10a440e784e7439dd6a1bfc8', 'request_id': 'req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:52.939299 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6 admin admin] The token cache is being invalidate because project 6d5e228f10a440e784e7439dd6a1bfc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:22:52.939636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:52.940986 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:52.946936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-b6bfd1d9-a46d-422a-a30e-c55ac7fb0895 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:52.947598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 850/1714] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:22:52 2022] DELETE /identity/v3/projects/6d5e228f10a440e784e7439dd6a1bfc8 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:52.948166 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b6bfd1d9-a46d-422a-a30e-c55ac7fb0895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:52.950322 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:52.952668 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:53.100666 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df2afc-0132-4bc5-a6ef-cf9f8e218dc6 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:53.109111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bc305-37f6-42d2-bf9a-b31c5b8c0e15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:53.110806 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bc305-37f6-42d2-bf9a-b31c5b8c0e15 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:53.111836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bc305-37f6-42d2-bf9a-b31c5b8c0e15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:53.112343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bc305-37f6-42d2-bf9a-b31c5b8c0e15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:53.123096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b05bc305-37f6-42d2-bf9a-b31c5b8c0e15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:53.183123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 865/1715] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:53.183997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfd1d9-a46d-422a-a30e-c55ac7fb0895 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:53.191122 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b6bfd1d9-a46d-422a-a30e-c55ac7fb0895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:53.202160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:53.220480 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfd1d9-a46d-422a-a30e-c55ac7fb0895 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:53.232770 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e349ee-8e79-4409-b953-7bbda7b9bccc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:53.233722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e349ee-8e79-4409-b953-7bbda7b9bccc service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:53.233981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e349ee-8e79-4409-b953-7bbda7b9bccc service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:53.234265 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e349ee-8e79-4409-b953-7bbda7b9bccc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:53.239068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-07e349ee-8e79-4409-b953-7bbda7b9bccc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:53.275500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05bc305-37f6-42d2-bf9a-b31c5b8c0e15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:53.277055 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05bc305-37f6-42d2-bf9a-b31c5b8c0e15 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:53.307651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 851/1716] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:53.309262 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b00627-21bf-4da2-93c4-e569b50bfa78 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:53.309744 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b00627-21bf-4da2-93c4-e569b50bfa78 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:53.310092 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b00627-21bf-4da2-93c4-e569b50bfa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:53.338116 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:53.360440 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-d5b00627-21bf-4da2-93c4-e569b50bfa78 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:53.362812 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d5b00627-21bf-4da2-93c4-e569b50bfa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:53.385515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 852/1717] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:53.386914 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0035b-67a1-4fd6-a719-aeb5bfde0974 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:53.387278 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0035b-67a1-4fd6-a719-aeb5bfde0974 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:53.387630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0035b-67a1-4fd6-a719-aeb5bfde0974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:53.433484 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-fab0035b-67a1-4fd6-a719-aeb5bfde0974 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:53.434200 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e349ee-8e79-4409-b953-7bbda7b9bccc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:53.434947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-fab0035b-67a1-4fd6-a719-aeb5bfde0974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:53.435654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e349ee-8e79-4409-b953-7bbda7b9bccc service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:53.456973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 853/1718] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:53.459408 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb628534-2f4b-4756-9e03-fa4b03489392 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:53.459694 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb628534-2f4b-4756-9e03-fa4b03489392 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:53.459964 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb628534-2f4b-4756-9e03-fa4b03489392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:53.466535 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 866/1719] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:53.467970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda14e-561f-4791-9694-f8306e55a8c3 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:53.468260 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda14e-561f-4791-9694-f8306e55a8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:53.468539 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda14e-561f-4791-9694-f8306e55a8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:53.526782 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-90eda14e-561f-4791-9694-f8306e55a8c3 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:53.528028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-90eda14e-561f-4791-9694-f8306e55a8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:53.552729 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 867/1720] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:53.554202 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eda14e-561f-4791-9694-f8306e55a8c3 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:53.560174 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-fb628534-2f4b-4756-9e03-fa4b03489392 None None] MFA Rules not processed for user `ef81663b621e458899ad0eaa324a81fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:53.566706 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eda14e-561f-4791-9694-f8306e55a8c3 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:53.569261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-fb628534-2f4b-4756-9e03-fa4b03489392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:53.573683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508bc74-9327-411d-94a7-33a0a086f53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:53.575088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508bc74-9327-411d-94a7-33a0a086f53a admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:53.575523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508bc74-9327-411d-94a7-33a0a086f53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:53.575985 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508bc74-9327-411d-94a7-33a0a086f53a admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:53.576789 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508bc74-9327-411d-94a7-33a0a086f53a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:22:53.577742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508bc74-9327-411d-94a7-33a0a086f53a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:53.579518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508bc74-9327-411d-94a7-33a0a086f53a admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:53.613268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 868/1721] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:22:53.616288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508bc74-9327-411d-94a7-33a0a086f53a admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:53.628083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508bc74-9327-411d-94a7-33a0a086f53a admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:53.634245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84375d00-9cda-4cb9-b4e1-761edcb97490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:53.634862 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84375d00-9cda-4cb9-b4e1-761edcb97490 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:53.635293 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84375d00-9cda-4cb9-b4e1-761edcb97490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:53.635576 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84375d00-9cda-4cb9-b4e1-761edcb97490 admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:53.636529 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84375d00-9cda-4cb9-b4e1-761edcb97490 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:22:53.636529 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84375d00-9cda-4cb9-b4e1-761edcb97490 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:53.639927 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84375d00-9cda-4cb9-b4e1-761edcb97490 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:53.662360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 869/1722] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:22:53.664699 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84375d00-9cda-4cb9-b4e1-761edcb97490 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:53.694167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84375d00-9cda-4cb9-b4e1-761edcb97490 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:53.709614 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec308057-59ec-4caf-9d71-2581ea3eeea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:53.710714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec308057-59ec-4caf-9d71-2581ea3eeea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:53.711783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec308057-59ec-4caf-9d71-2581ea3eeea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:53.713508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec308057-59ec-4caf-9d71-2581ea3eeea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:53.716037 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec308057-59ec-4caf-9d71-2581ea3eeea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:53.720818 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec308057-59ec-4caf-9d71-2581ea3eeea2 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:53.736775 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 854/1723] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:53 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:53.741693 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb628534-2f4b-4756-9e03-fa4b03489392 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:53.788742 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb628534-2f4b-4756-9e03-fa4b03489392 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:53.794145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 870/1724] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:53 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:53.794145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec308057-59ec-4caf-9d71-2581ea3eeea2 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:53.959826 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec308057-59ec-4caf-9d71-2581ea3eeea2 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:53.976231 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0207be-42a6-46ea-92eb-14307041c1db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:53.979338 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0207be-42a6-46ea-92eb-14307041c1db service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:53.979634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0207be-42a6-46ea-92eb-14307041c1db service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:53.980063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0207be-42a6-46ea-92eb-14307041c1db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:53.986986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-df0207be-42a6-46ea-92eb-14307041c1db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:54.001880 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1744334-092d-4bab-92d5-44cd3bf6321b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:54.002579 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1744334-092d-4bab-92d5-44cd3bf6321b admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:54.002985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1744334-092d-4bab-92d5-44cd3bf6321b admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:54.003264 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1744334-092d-4bab-92d5-44cd3bf6321b admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:54.003910 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1744334-092d-4bab-92d5-44cd3bf6321b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:54.010540 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1744334-092d-4bab-92d5-44cd3bf6321b admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:54.091102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 855/1725] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:53 2022] POST /identity/v3/projects => generated 389 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:54.091102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1744334-092d-4bab-92d5-44cd3bf6321b admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:54.095633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f1744334-092d-4bab-92d5-44cd3bf6321b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:54.164046 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0207be-42a6-46ea-92eb-14307041c1db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:54.170137 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0207be-42a6-46ea-92eb-14307041c1db service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:54.206565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 871/1726] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:54.208756 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bba56-7b80-4dfe-bc97-23d67b6842bd None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:54.209813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bba56-7b80-4dfe-bc97-23d67b6842bd None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:54.210277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bba56-7b80-4dfe-bc97-23d67b6842bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:54.218583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:54.293162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-015bba56-7b80-4dfe-bc97-23d67b6842bd None None] MFA Rules not processed for user `8429a502bca744fc9f5d5701fe9c766c`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:54.294534 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-015bba56-7b80-4dfe-bc97-23d67b6842bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:54.319758 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1744334-092d-4bab-92d5-44cd3bf6321b admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:54.330659 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc135bb1-dbdc-4bd5-9a4f-eaf2e7835dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:54.331331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc135bb1-dbdc-4bd5-9a4f-eaf2e7835dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:54.331623 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc135bb1-dbdc-4bd5-9a4f-eaf2e7835dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:54.333414 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc135bb1-dbdc-4bd5-9a4f-eaf2e7835dd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:54.335191 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc135bb1-dbdc-4bd5-9a4f-eaf2e7835dd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:54.341438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc135bb1-dbdc-4bd5-9a4f-eaf2e7835dd3 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:54.364072 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:54.419936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 856/1727] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:54 2022] POST /identity/v3/projects => generated 405 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:54.426013 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc135bb1-dbdc-4bd5-9a4f-eaf2e7835dd3 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:54.507683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 872/1728] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:54.508746 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015bba56-7b80-4dfe-bc97-23d67b6842bd None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:54.537700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015bba56-7b80-4dfe-bc97-23d67b6842bd None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:54.569856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc135bb1-dbdc-4bd5-9a4f-eaf2e7835dd3 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:54.579750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174d418-0311-4fbe-986f-83dac97245e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:54.580983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174d418-0311-4fbe-986f-83dac97245e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:54.581359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174d418-0311-4fbe-986f-83dac97245e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:54.581925 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174d418-0311-4fbe-986f-83dac97245e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:54.582724 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174d418-0311-4fbe-986f-83dac97245e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:54.586488 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174d418-0311-4fbe-986f-83dac97245e1 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:54.637860 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 857/1729] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:54 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:54.637860 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174d418-0311-4fbe-986f-83dac97245e1 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:54.655171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174d418-0311-4fbe-986f-83dac97245e1 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:54.655573 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47b35b-7510-434b-86e4-d0689a06de8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:54.656358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47b35b-7510-434b-86e4-d0689a06de8c admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:54.656755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47b35b-7510-434b-86e4-d0689a06de8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:54.657120 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b47b35b-7510-434b-86e4-d0689a06de8c admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:54.658048 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b47b35b-7510-434b-86e4-d0689a06de8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:54.660203 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b47b35b-7510-434b-86e4-d0689a06de8c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:54.675930 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d7294-5d21-44e2-a5a1-4dc0a56ce11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:54.678225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d7294-5d21-44e2-a5a1-4dc0a56ce11c admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:54.678729 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d7294-5d21-44e2-a5a1-4dc0a56ce11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:54.679215 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d7294-5d21-44e2-a5a1-4dc0a56ce11c admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:54.680836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d7294-5d21-44e2-a5a1-4dc0a56ce11c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:54.683914 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d7294-5d21-44e2-a5a1-4dc0a56ce11c admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:54.691838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 873/1730] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:54 2022] POST /identity/v3/users => generated 295 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:54.691838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47b35b-7510-434b-86e4-d0689a06de8c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:54.696040 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5b47b35b-7510-434b-86e4-d0689a06de8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:54.719415 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 858/1731] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:54 2022] POST /identity/v3/users => generated 309 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:54.720419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d7294-5d21-44e2-a5a1-4dc0a56ce11c admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:54.736660 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d7294-5d21-44e2-a5a1-4dc0a56ce11c admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:54.748937 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85477e7-b6e0-44a1-a0e4-d01746dc28e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:54.750697 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85477e7-b6e0-44a1-a0e4-d01746dc28e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:54.751218 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85477e7-b6e0-44a1-a0e4-d01746dc28e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:54.752804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85477e7-b6e0-44a1-a0e4-d01746dc28e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:54.754325 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85477e7-b6e0-44a1-a0e4-d01746dc28e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:54.758964 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85477e7-b6e0-44a1-a0e4-d01746dc28e2 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:54.778167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 859/1732] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:54.779836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834f5a7-bbf6-463a-b838-9d1737497d27 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:54.780458 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834f5a7-bbf6-463a-b838-9d1737497d27 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:54.781059 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834f5a7-bbf6-463a-b838-9d1737497d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:54.851278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b47b35b-7510-434b-86e4-d0689a06de8c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:54.867949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-5834f5a7-bbf6-463a-b838-9d1737497d27 None None] MFA Rules not processed for user `95575c3357d4413298c3fa08709c5037`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:54.869165 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c7591-82c8-4ae6-af92-a9dfa8d8ab83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:54.870087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c7591-82c8-4ae6-af92-a9dfa8d8ab83 service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:54.870513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c7591-82c8-4ae6-af92-a9dfa8d8ab83 service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:54.870785 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c7591-82c8-4ae6-af92-a9dfa8d8ab83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:54.874907 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5834f5a7-bbf6-463a-b838-9d1737497d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:55.000923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:55.008781 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 860/1733] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:55.009793 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834f5a7-bbf6-463a-b838-9d1737497d27 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:55.032414 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834f5a7-bbf6-463a-b838-9d1737497d27 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:55.033912 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c7591-82c8-4ae6-af92-a9dfa8d8ab83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:55.040253 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c7591-82c8-4ae6-af92-a9dfa8d8ab83 service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:55.048454 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8cde4-534c-4394-86b4-576fa0003ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:55.049087 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8cde4-534c-4394-86b4-576fa0003ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:55.049461 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8cde4-534c-4394-86b4-576fa0003ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:55.049737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8cde4-534c-4394-86b4-576fa0003ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:55.050567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8cde4-534c-4394-86b4-576fa0003ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:55.053101 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8cde4-534c-4394-86b4-576fa0003ea6 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:55.054816 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:55.091889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 861/1734] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:55.093950 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 874/1735] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:55.096089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8cde4-534c-4394-86b4-576fa0003ea6 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:55.097187 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c7591-82c8-4ae6-af92-a9dfa8d8ab83 service nova] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:55.125181 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8cde4-534c-4394-86b4-576fa0003ea6 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:55.139534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c6134f-a729-4e64-83f1-121c1963a70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:55.140411 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6134f-a729-4e64-83f1-121c1963a70c admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:55.140833 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6134f-a729-4e64-83f1-121c1963a70c admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:55.141272 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c6134f-a729-4e64-83f1-121c1963a70c admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:55.142093 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6134f-a729-4e64-83f1-121c1963a70c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:55.145962 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c6134f-a729-4e64-83f1-121c1963a70c admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:55.157848 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c7591-82c8-4ae6-af92-a9dfa8d8ab83 service nova] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:55.183777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 862/1736] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:55.184277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390a8bc-734c-41cc-942d-5e0523ea58f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:55.185023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390a8bc-734c-41cc-942d-5e0523ea58f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:55.185936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb8e04-4fb7-45b8-b4d1-cadd894694a6 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:55.186226 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb8e04-4fb7-45b8-b4d1-cadd894694a6 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:55.186502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb8e04-4fb7-45b8-b4d1-cadd894694a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:55.186903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390a8bc-734c-41cc-942d-5e0523ea58f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:55.190582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390a8bc-734c-41cc-942d-5e0523ea58f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:55.191429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390a8bc-734c-41cc-942d-5e0523ea58f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:55.209172 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390a8bc-734c-41cc-942d-5e0523ea58f3 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:55.227325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 875/1737] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:55.228338 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390a8bc-734c-41cc-942d-5e0523ea58f3 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:55.252462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-c9cb8e04-4fb7-45b8-b4d1-cadd894694a6 None None] MFA Rules not processed for user `95575c3357d4413298c3fa08709c5037`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:55.259174 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c9cb8e04-4fb7-45b8-b4d1-cadd894694a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:55.264672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390a8bc-734c-41cc-942d-5e0523ea58f3 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:55.282111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e8bcd-174e-446e-b0af-c65dcc6c57ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:55.284142 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e8bcd-174e-446e-b0af-c65dcc6c57ad admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:55.284501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e8bcd-174e-446e-b0af-c65dcc6c57ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:55.285962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e8bcd-174e-446e-b0af-c65dcc6c57ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:55.290896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e8bcd-174e-446e-b0af-c65dcc6c57ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:55.292115 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 863/1738] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:55.293074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb8e04-4fb7-45b8-b4d1-cadd894694a6 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:55.300597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e8bcd-174e-446e-b0af-c65dcc6c57ad admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:55.331992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb8e04-4fb7-45b8-b4d1-cadd894694a6 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:55.347671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 876/1739] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:55.350736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e8bcd-174e-446e-b0af-c65dcc6c57ad admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:55.351921 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fb4f2-d08e-4878-8cd7-a325ab11d72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:55.360749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fb4f2-d08e-4878-8cd7-a325ab11d72f admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:55.361157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fb4f2-d08e-4878-8cd7-a325ab11d72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:55.361667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fb4f2-d08e-4878-8cd7-a325ab11d72f admin admin] PATH_INFO: `/v3/projects/145957776a4a45c183c7569ecef1b245/users/cca22bda316748929f5358d6e1f95fef/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:55.382692 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e8bcd-174e-446e-b0af-c65dcc6c57ad admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:55.396563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c827cd1-8c9f-444a-923c-d16bfeac5f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:55.397932 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fb4f2-d08e-4878-8cd7-a325ab11d72f admin admin] RBAC: Authorizing `identity:create_grant(project_id=145957776a4a45c183c7569ecef1b245, user_id=cca22bda316748929f5358d6e1f95fef, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:55.400715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c827cd1-8c9f-444a-923c-d16bfeac5f90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:55.400843 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fb4f2-d08e-4878-8cd7-a325ab11d72f admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:55.401414 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c827cd1-8c9f-444a-923c-d16bfeac5f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:55.402681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c827cd1-8c9f-444a-923c-d16bfeac5f90 admin admin] PATH_INFO: `/v3/projects/0fe166ee1eb64abfafe330a705e3b03d/users/2ce30508a72f478b933e4bcc0a4153a9/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:55.443286 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 864/1740] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:55 2022] PUT /identity/v3/projects/145957776a4a45c183c7569ecef1b245/users/cca22bda316748929f5358d6e1f95fef/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:55.446256 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fb4f2-d08e-4878-8cd7-a325ab11d72f admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:55.451763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-dc0fb4f2-d08e-4878-8cd7-a325ab11d72f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:55.461671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c827cd1-8c9f-444a-923c-d16bfeac5f90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe166ee1eb64abfafe330a705e3b03d, user_id=2ce30508a72f478b933e4bcc0a4153a9, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:55.467512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c827cd1-8c9f-444a-923c-d16bfeac5f90 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:55.513022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 877/1741] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:55 2022] PUT /identity/v3/projects/0fe166ee1eb64abfafe330a705e3b03d/users/2ce30508a72f478b933e4bcc0a4153a9/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:55.514377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c827cd1-8c9f-444a-923c-d16bfeac5f90 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:55.774308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fb4f2-d08e-4878-8cd7-a325ab11d72f admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:55.842255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c827cd1-8c9f-444a-923c-d16bfeac5f90 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:55.862238 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66b1f2-1386-4f13-b6cd-967b21f479de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:55.863069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66b1f2-1386-4f13-b6cd-967b21f479de admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:55.863256 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66b1f2-1386-4f13-b6cd-967b21f479de admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:55.863515 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66b1f2-1386-4f13-b6cd-967b21f479de admin admin] PATH_INFO: `/v3/projects/7ac45189b8fd4821914bd589696ed26d/users/00fa5cb7f14e4e9899563bb5f1c7ca5e/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:55.909918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66b1f2-1386-4f13-b6cd-967b21f479de admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ac45189b8fd4821914bd589696ed26d, user_id=00fa5cb7f14e4e9899563bb5f1c7ca5e, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:55.915882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66b1f2-1386-4f13-b6cd-967b21f479de admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:55.952453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 878/1742] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:55 2022] PUT /identity/v3/projects/7ac45189b8fd4821914bd589696ed26d/users/00fa5cb7f14e4e9899563bb5f1c7ca5e/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:55.955308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66b1f2-1386-4f13-b6cd-967b21f479de admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:56.047589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b98bc0-aa22-43e4-97e2-7dff36ba5677 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:56.049732 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b98bc0-aa22-43e4-97e2-7dff36ba5677 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:56.050171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b98bc0-aa22-43e4-97e2-7dff36ba5677 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:56.051377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b98bc0-aa22-43e4-97e2-7dff36ba5677 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:56.145842 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66b1f2-1386-4f13-b6cd-967b21f479de admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:56.161554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2bcb5-f070-4cf4-9ac2-77aad3e466e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:56.163838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2bcb5-f070-4cf4-9ac2-77aad3e466e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:56.164262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2bcb5-f070-4cf4-9ac2-77aad3e466e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:56.164606 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2bcb5-f070-4cf4-9ac2-77aad3e466e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:56.172637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:56.177619 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2bcb5-f070-4cf4-9ac2-77aad3e466e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:56.199629 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2bcb5-f070-4cf4-9ac2-77aad3e466e3 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:56.215105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b98bc0-aa22-43e4-97e2-7dff36ba5677 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:56.221824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b98bc0-aa22-43e4-97e2-7dff36ba5677 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:56.222655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 879/1743] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:56.225930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6dea5-9e28-4b1c-8d14-74f82a228e9d None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:56.227875 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6dea5-9e28-4b1c-8d14-74f82a228e9d None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:56.227996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6dea5-9e28-4b1c-8d14-74f82a228e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:56.261086 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 865/1744] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 817 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:56.267728 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441ebd2-efab-49cb-a7ed-75d44496b2f2 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:56.268019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441ebd2-efab-49cb-a7ed-75d44496b2f2 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:56.268282 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441ebd2-efab-49cb-a7ed-75d44496b2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:56.304340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-a9d6dea5-9e28-4b1c-8d14-74f82a228e9d None None] MFA Rules not processed for user `2ce30508a72f478b933e4bcc0a4153a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:56.305551 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a9d6dea5-9e28-4b1c-8d14-74f82a228e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:56.326391 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:56.376628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e441ebd2-efab-49cb-a7ed-75d44496b2f2 None None] MFA Rules not processed for user `00fa5cb7f14e4e9899563bb5f1c7ca5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:56.377901 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e441ebd2-efab-49cb-a7ed-75d44496b2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:56.509223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 880/1745] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:56.511805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b51b75-e59c-43dc-9c18-93ffb07d5e42 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:56.512671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b51b75-e59c-43dc-9c18-93ffb07d5e42 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:56.512671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b51b75-e59c-43dc-9c18-93ffb07d5e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:56.558982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-c2b51b75-e59c-43dc-9c18-93ffb07d5e42 None None] MFA Rules not processed for user `4d7b7c68a699443aa5c0859094c265ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:56.564142 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-c2b51b75-e59c-43dc-9c18-93ffb07d5e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:56.589064 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 866/1746] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:56.589064 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441ebd2-efab-49cb-a7ed-75d44496b2f2 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:56.623520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441ebd2-efab-49cb-a7ed-75d44496b2f2 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:56.643032 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b67095-7dd0-4b07-86e1-f7ce8ba2ea76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:56.645434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b67095-7dd0-4b07-86e1-f7ce8ba2ea76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:56.645790 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b67095-7dd0-4b07-86e1-f7ce8ba2ea76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:56.649492 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b67095-7dd0-4b07-86e1-f7ce8ba2ea76 admin admin] PATH_INFO: `/v3/projects/145957776a4a45c183c7569ecef1b245/users/cca22bda316748929f5358d6e1f95fef/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:56.658990 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b67095-7dd0-4b07-86e1-f7ce8ba2ea76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=145957776a4a45c183c7569ecef1b245, user_id=cca22bda316748929f5358d6e1f95fef, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:56.669515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b67095-7dd0-4b07-86e1-f7ce8ba2ea76 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:56.701605 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 867/1747] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:56 2022] PUT /identity/v3/projects/145957776a4a45c183c7569ecef1b245/users/cca22bda316748929f5358d6e1f95fef/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:56.702460 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b67095-7dd0-4b07-86e1-f7ce8ba2ea76 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:56.728196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 881/1748] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:56.729177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b51b75-e59c-43dc-9c18-93ffb07d5e42 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:56.793028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b51b75-e59c-43dc-9c18-93ffb07d5e42 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:56.887850 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b67095-7dd0-4b07-86e1-f7ce8ba2ea76 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:56.900769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47148c1a-a471-4d88-856d-b6472078b040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:56.901710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47148c1a-a471-4d88-856d-b6472078b040 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:56.902070 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47148c1a-a471-4d88-856d-b6472078b040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:56.902643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47148c1a-a471-4d88-856d-b6472078b040 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:56.903567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47148c1a-a471-4d88-856d-b6472078b040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:56.909971 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47148c1a-a471-4d88-856d-b6472078b040 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:56.938863 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc8ec9-aa3b-4655-a67d-e76d9339fc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:56.939810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc8ec9-aa3b-4655-a67d-e76d9339fc9b admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:56.940190 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc8ec9-aa3b-4655-a67d-e76d9339fc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:56.944859 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc8ec9-aa3b-4655-a67d-e76d9339fc9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:56.945838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc8ec9-aa3b-4655-a67d-e76d9339fc9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:56.948258 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc8ec9-aa3b-4655-a67d-e76d9339fc9b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:56.983153 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 868/1749] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:56 2022] POST /identity/v3/projects => generated 377 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:56.984123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47148c1a-a471-4d88-856d-b6472078b040 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:57.024885 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 882/1750] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:22:56 2022] POST /identity/v3/projects => generated 389 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:57.026319 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc8ec9-aa3b-4655-a67d-e76d9339fc9b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:57.103040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:57.124969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:57.128851 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47148c1a-a471-4d88-856d-b6472078b040 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:57.173308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc8ec9-aa3b-4655-a67d-e76d9339fc9b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:57.188114 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca526b38-c1da-4c4c-a0d2-ef8554b8ef7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:57.189113 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca526b38-c1da-4c4c-a0d2-ef8554b8ef7b admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:57.189611 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca526b38-c1da-4c4c-a0d2-ef8554b8ef7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:57.190072 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca526b38-c1da-4c4c-a0d2-ef8554b8ef7b admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:57.190896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca526b38-c1da-4c4c-a0d2-ef8554b8ef7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:57.194284 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca526b38-c1da-4c4c-a0d2-ef8554b8ef7b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:57.220966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 883/1751] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:57 2022] POST /identity/v3/users => generated 301 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:57.225768 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca526b38-c1da-4c4c-a0d2-ef8554b8ef7b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:57.240951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca526b38-c1da-4c4c-a0d2-ef8554b8ef7b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:57.246273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59d914-5c09-4e5d-8ca3-23beefd8d980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:57.247065 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59d914-5c09-4e5d-8ca3-23beefd8d980 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:57.247484 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59d914-5c09-4e5d-8ca3-23beefd8d980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:57.247949 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59d914-5c09-4e5d-8ca3-23beefd8d980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:57.248730 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59d914-5c09-4e5d-8ca3-23beefd8d980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:57.250836 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59d914-5c09-4e5d-8ca3-23beefd8d980 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:57.268391 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 884/1752] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:57.272323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59d914-5c09-4e5d-8ca3-23beefd8d980 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:57.287815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59d914-5c09-4e5d-8ca3-23beefd8d980 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:57.295141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de81320-2259-46c1-993c-61398b30d4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:57.295769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de81320-2259-46c1-993c-61398b30d4d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:57.296048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de81320-2259-46c1-993c-61398b30d4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:57.297409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de81320-2259-46c1-993c-61398b30d4d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:57.298058 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de81320-2259-46c1-993c-61398b30d4d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:57.299983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de81320-2259-46c1-993c-61398b30d4d6 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:57.306514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736f81d-7e8a-49cf-80f2-6fa3d5062dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:57.307624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736f81d-7e8a-49cf-80f2-6fa3d5062dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:57.307779 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736f81d-7e8a-49cf-80f2-6fa3d5062dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:57.308465 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736f81d-7e8a-49cf-80f2-6fa3d5062dbc admin admin] PATH_INFO: `/v3/projects/879c3e76c79044568d4bb2993e6e8b97/users/fb43dfcacb13409a90dd76417f4e07cd/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:57.346040 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736f81d-7e8a-49cf-80f2-6fa3d5062dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=879c3e76c79044568d4bb2993e6e8b97, user_id=fb43dfcacb13409a90dd76417f4e07cd, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:57.356718 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 869/1753] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:22:56 2022] POST /identity/v3/users => generated 295 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:22:57.357083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736f81d-7e8a-49cf-80f2-6fa3d5062dbc admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:57.359469 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de81320-2259-46c1-993c-61398b30d4d6 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:57.386442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de81320-2259-46c1-993c-61398b30d4d6 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:57.389670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 885/1754] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:57 2022] PUT /identity/v3/projects/879c3e76c79044568d4bb2993e6e8b97/users/fb43dfcacb13409a90dd76417f4e07cd/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:57.392341 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736f81d-7e8a-49cf-80f2-6fa3d5062dbc admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:57.532751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e4afe-3805-43c9-b283-abf79370082a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:57.534265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e4afe-3805-43c9-b283-abf79370082a admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:57.534718 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e4afe-3805-43c9-b283-abf79370082a admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:57.535092 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e4afe-3805-43c9-b283-abf79370082a admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:57.537157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e4afe-3805-43c9-b283-abf79370082a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:57.538961 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e4afe-3805-43c9-b283-abf79370082a admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:57.573052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736f81d-7e8a-49cf-80f2-6fa3d5062dbc admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:57.586043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ff83e-5170-4e7f-8573-a04fd3e701c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:57.586634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ff83e-5170-4e7f-8573-a04fd3e701c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:57.586916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ff83e-5170-4e7f-8573-a04fd3e701c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:57.588167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ff83e-5170-4e7f-8573-a04fd3e701c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:57.588167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ff83e-5170-4e7f-8573-a04fd3e701c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:57.592333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 870/1755] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:57.593291 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ff83e-5170-4e7f-8573-a04fd3e701c7 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:57.595721 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e4afe-3805-43c9-b283-abf79370082a admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:57.646008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e4afe-3805-43c9-b283-abf79370082a admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:57.648296 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 886/1756] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:57.655073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ff83e-5170-4e7f-8573-a04fd3e701c7 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:57.666039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dffb1b-0f87-4f3a-b69b-cfc60a9db29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:57.667826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dffb1b-0f87-4f3a-b69b-cfc60a9db29e admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:57.668029 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dffb1b-0f87-4f3a-b69b-cfc60a9db29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:57.668308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dffb1b-0f87-4f3a-b69b-cfc60a9db29e admin admin] PATH_INFO: `/v3/projects/d58cea1eda854e9580a6a3839a45c4a5/users/c66ea5585c7b4adcbf5afab7a911d064/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:57.671357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ff83e-5170-4e7f-8573-a04fd3e701c7 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:57.679329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd94ac0-b599-4365-b206-083d2d65ca71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:57.679703 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd94ac0-b599-4365-b206-083d2d65ca71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:57.679996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd94ac0-b599-4365-b206-083d2d65ca71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:57.680280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd94ac0-b599-4365-b206-083d2d65ca71 admin admin] PATH_INFO: `/v3/projects/879c3e76c79044568d4bb2993e6e8b97/users/fb43dfcacb13409a90dd76417f4e07cd/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:57.682318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd94ac0-b599-4365-b206-083d2d65ca71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=879c3e76c79044568d4bb2993e6e8b97, user_id=fb43dfcacb13409a90dd76417f4e07cd, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:57.685066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd94ac0-b599-4365-b206-083d2d65ca71 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:57.705272 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dffb1b-0f87-4f3a-b69b-cfc60a9db29e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d58cea1eda854e9580a6a3839a45c4a5, user_id=c66ea5585c7b4adcbf5afab7a911d064, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:57.715916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 887/1757] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:57 2022] PUT /identity/v3/projects/879c3e76c79044568d4bb2993e6e8b97/users/fb43dfcacb13409a90dd76417f4e07cd/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:57.716383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dffb1b-0f87-4f3a-b69b-cfc60a9db29e admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:57.717843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fa29b-9144-4ff0-b39d-32be20085e62 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:57.718197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fa29b-9144-4ff0-b39d-32be20085e62 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:57.718533 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fa29b-9144-4ff0-b39d-32be20085e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:57.742021 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 871/1758] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:57 2022] PUT /identity/v3/projects/d58cea1eda854e9580a6a3839a45c4a5/users/c66ea5585c7b4adcbf5afab7a911d064/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:57.746253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dffb1b-0f87-4f3a-b69b-cfc60a9db29e admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:57.793044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-788fa29b-9144-4ff0-b39d-32be20085e62 None None] MFA Rules not processed for user `fb43dfcacb13409a90dd76417f4e07cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:57.794755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-788fa29b-9144-4ff0-b39d-32be20085e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:57.926701 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dffb1b-0f87-4f3a-b69b-cfc60a9db29e admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:57.945488 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 888/1759] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:57 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:57.950114 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15016ae4-1bfc-4a43-8f08-3eb460a2cba9 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:57.950486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15016ae4-1bfc-4a43-8f08-3eb460a2cba9 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:57.950900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15016ae4-1bfc-4a43-8f08-3eb460a2cba9 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:57.951212 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd17c6-245f-48fd-a1bc-f66e2328983d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:57.951842 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd17c6-245f-48fd-a1bc-f66e2328983d admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:57.952121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd17c6-245f-48fd-a1bc-f66e2328983d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:57.952393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd17c6-245f-48fd-a1bc-f66e2328983d admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:57.953567 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 889/1760] 10.222.0.60 () {54 vars in 912 bytes} [Wed Sep 28 12:22:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:22:57.955290 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd17c6-245f-48fd-a1bc-f66e2328983d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:57.959536 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15016ae4-1bfc-4a43-8f08-3eb460a2cba9 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:57.963991 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd17c6-245f-48fd-a1bc-f66e2328983d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:57.967089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:57.970634 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:58.003112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 872/1761] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:22:58.011250 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36ee3f-2efa-4c2a-8c96-f3ecbbef29dd None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:58.017057 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36ee3f-2efa-4c2a-8c96-f3ecbbef29dd None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:58.017775 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36ee3f-2efa-4c2a-8c96-f3ecbbef29dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:58.169115 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-bd36ee3f-2efa-4c2a-8c96-f3ecbbef29dd None None] MFA Rules not processed for user `ba75586d3a7d4278ad6c0c31a1bed3df`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:58.170480 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-bd36ee3f-2efa-4c2a-8c96-f3ecbbef29dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:58.180123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15016ae4-1bfc-4a43-8f08-3eb460a2cba9 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:58.202757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90fcb1-62c9-4ded-a32c-8af3cfe4c5f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:58.203209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90fcb1-62c9-4ded-a32c-8af3cfe4c5f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:58.204700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90fcb1-62c9-4ded-a32c-8af3cfe4c5f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:58.204985 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90fcb1-62c9-4ded-a32c-8af3cfe4c5f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:58.248092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90fcb1-62c9-4ded-a32c-8af3cfe4c5f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:58.249930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90fcb1-62c9-4ded-a32c-8af3cfe4c5f0 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:58.302814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 890/1762] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:57 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:58.304141 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90fcb1-62c9-4ded-a32c-8af3cfe4c5f0 service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:58.368738 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 873/1763] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:58.369869 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffff18-dd10-4f76-a94a-3d2a31a5f84a None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:58.370163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffff18-dd10-4f76-a94a-3d2a31a5f84a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:58.370903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90fcb1-62c9-4ded-a32c-8af3cfe4c5f0 service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:58.372546 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffff18-dd10-4f76-a94a-3d2a31a5f84a None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:58.375879 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 874/1764] 10.222.0.60 () {54 vars in 912 bytes} [Wed Sep 28 12:22:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:22:58.381734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4121e-6683-4295-ac9d-f18f312c788e None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:58.381734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4121e-6683-4295-ac9d-f18f312c788e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:58.382107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4121e-6683-4295-ac9d-f18f312c788e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:58.393371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c94145-9fb9-430c-8953-77703cdd8dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:58.394633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c94145-9fb9-430c-8953-77703cdd8dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:58.394633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c94145-9fb9-430c-8953-77703cdd8dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:58.394839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c94145-9fb9-430c-8953-77703cdd8dc2 admin admin] PATH_INFO: `/v3/projects/d58cea1eda854e9580a6a3839a45c4a5/users/c66ea5585c7b4adcbf5afab7a911d064/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:58.410584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c94145-9fb9-430c-8953-77703cdd8dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d58cea1eda854e9580a6a3839a45c4a5, user_id=c66ea5585c7b4adcbf5afab7a911d064, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:58.422103 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c94145-9fb9-430c-8953-77703cdd8dc2 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:58.448689 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-50f4121e-6683-4295-ac9d-f18f312c788e None None] MFA Rules not processed for user `ba75586d3a7d4278ad6c0c31a1bed3df`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:58.449606 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 891/1765] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:22:58 2022] PUT /identity/v3/projects/d58cea1eda854e9580a6a3839a45c4a5/users/c66ea5585c7b4adcbf5afab7a911d064/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:58.463040 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439f013-62da-4055-bd98-c1ed3b9e171f None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:58.463327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439f013-62da-4055-bd98-c1ed3b9e171f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:58.463606 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439f013-62da-4055-bd98-c1ed3b9e171f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:58.478134 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-50f4121e-6683-4295-ac9d-f18f312c788e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:58.606216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-2439f013-62da-4055-bd98-c1ed3b9e171f None None] MFA Rules not processed for user `c66ea5585c7b4adcbf5afab7a911d064`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:58.607633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2439f013-62da-4055-bd98-c1ed3b9e171f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:58.697457 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 875/1766] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:58.700163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4121e-6683-4295-ac9d-f18f312c788e None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:58.704055 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-50f4121e-6683-4295-ac9d-f18f312c788e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:58.728911 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4121e-6683-4295-ac9d-f18f312c788e None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:58.738137 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf7d79-0f31-4403-985d-834dbafcfd3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:58.739035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf7d79-0f31-4403-985d-834dbafcfd3d service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:58.739035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf7d79-0f31-4403-985d-834dbafcfd3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:58.739319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf7d79-0f31-4403-985d-834dbafcfd3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:58.763461 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf7d79-0f31-4403-985d-834dbafcfd3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:58.765029 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf7d79-0f31-4403-985d-834dbafcfd3d service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:58.778321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 876/1767] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:58.779241 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf7d79-0f31-4403-985d-834dbafcfd3d service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:58.812544 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 892/1768] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:58.827807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439f013-62da-4055-bd98-c1ed3b9e171f None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:58.830227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2439f013-62da-4055-bd98-c1ed3b9e171f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:58.870339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439f013-62da-4055-bd98-c1ed3b9e171f None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:58.884063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f75868-85f6-41ea-875a-20ee8c9f2c74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:58.884727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f75868-85f6-41ea-875a-20ee8c9f2c74 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:58.888108 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f75868-85f6-41ea-875a-20ee8c9f2c74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:58.888471 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f75868-85f6-41ea-875a-20ee8c9f2c74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:58.908344 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f75868-85f6-41ea-875a-20ee8c9f2c74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:58.909856 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f75868-85f6-41ea-875a-20ee8c9f2c74 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:58.916794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf7d79-0f31-4403-985d-834dbafcfd3d service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:58.924720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf0031-8ddd-4eb2-81f4-90c5e8233b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:58.930163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 893/1769] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:58.931710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b78704-8d8c-435b-94fe-a88bd461a1dd None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:58.932053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b78704-8d8c-435b-94fe-a88bd461a1dd None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:58.932334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b78704-8d8c-435b-94fe-a88bd461a1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:58.960203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf0031-8ddd-4eb2-81f4-90c5e8233b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:58.960496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf0031-8ddd-4eb2-81f4-90c5e8233b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:58.960778 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf0031-8ddd-4eb2-81f4-90c5e8233b83 admin admin] PATH_INFO: `/v3/users/00fa5cb7f14e4e9899563bb5f1c7ca5e` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:58.969890 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf0031-8ddd-4eb2-81f4-90c5e8233b83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00fa5cb7f14e4e9899563bb5f1c7ca5e)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:58.977232 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf0031-8ddd-4eb2-81f4-90c5e8233b83 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:58.987948 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-11b78704-8d8c-435b-94fe-a88bd461a1dd None None] MFA Rules not processed for user `cca22bda316748929f5358d6e1f95fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:58.990503 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-11b78704-8d8c-435b-94fe-a88bd461a1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:59.074679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:59.128736 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-4dbf0031-8ddd-4eb2-81f4-90c5e8233b83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00fa5cb7f14e4e9899563bb5f1c7ca5e'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:59.129481 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-4dbf0031-8ddd-4eb2-81f4-90c5e8233b83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00fa5cb7f14e4e9899563bb5f1c7ca5e'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:59.159804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:22:59.167061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-4dbf0031-8ddd-4eb2-81f4-90c5e8233b83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00fa5cb7f14e4e9899563bb5f1c7ca5e'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:59.182751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-4dbf0031-8ddd-4eb2-81f4-90c5e8233b83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00fa5cb7f14e4e9899563bb5f1c7ca5e'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:59.183439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-4dbf0031-8ddd-4eb2-81f4-90c5e8233b83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00fa5cb7f14e4e9899563bb5f1c7ca5e'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:22:59.223174 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 877/1770] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:22:58 2022] DELETE /identity/v3/users/00fa5cb7f14e4e9899563bb5f1c7ca5e => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:22:59.225450 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf0031-8ddd-4eb2-81f4-90c5e8233b83 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:59.226307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 894/1771] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:59.234046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-4dbf0031-8ddd-4eb2-81f4-90c5e8233b83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:59.234778 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c527b-b6d0-415a-893f-2793774d0679 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:59.235079 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c527b-b6d0-415a-893f-2793774d0679 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:59.235361 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c527b-b6d0-415a-893f-2793774d0679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:59.260000 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-089c527b-b6d0-415a-893f-2793774d0679 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:59.261409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-089c527b-b6d0-415a-893f-2793774d0679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:59.398463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 895/1772] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:22:59.405324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf0031-8ddd-4eb2-81f4-90c5e8233b83 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:59.412723 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f0dde-24b3-4a08-9980-07c94fce76e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:59.413648 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f0dde-24b3-4a08-9980-07c94fce76e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:59.414044 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f0dde-24b3-4a08-9980-07c94fce76e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:59.414404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f0dde-24b3-4a08-9980-07c94fce76e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:59.417636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1b6f0dde-24b3-4a08-9980-07c94fce76e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:59.480923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c527b-b6d0-415a-893f-2793774d0679 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:22:59.482502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-089c527b-b6d0-415a-893f-2793774d0679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:22:59.560062 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f0dde-24b3-4a08-9980-07c94fce76e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:59.561994 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f0dde-24b3-4a08-9980-07c94fce76e7 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:59.575549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 878/1773] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:59.601289 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c527b-b6d0-415a-893f-2793774d0679 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:22:59.615391 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c749ce-15b3-4802-9952-090558ef396d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:22:59.616040 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c749ce-15b3-4802-9952-090558ef396d service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:59.616650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c749ce-15b3-4802-9952-090558ef396d service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:59.616767 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c749ce-15b3-4802-9952-090558ef396d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:59.637913 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c749ce-15b3-4802-9952-090558ef396d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:22:59.639363 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c749ce-15b3-4802-9952-090558ef396d service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:22:59.660411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 896/1774] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:22:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:22:59.820584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed0729-d7f2-42d5-b396-4d22dfa92b43 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:22:59.820874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed0729-d7f2-42d5-b396-4d22dfa92b43 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:22:59.821170 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed0729-d7f2-42d5-b396-4d22dfa92b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:22:59.867049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-46ed0729-d7f2-42d5-b396-4d22dfa92b43 None None] MFA Rules not processed for user `8429a502bca744fc9f5d5701fe9c766c`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:22:59.868473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-46ed0729-d7f2-42d5-b396-4d22dfa92b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:00.066656 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 879/1775] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:22:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:00.207631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c749ce-15b3-4802-9952-090558ef396d service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:00.210217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-74c749ce-15b3-4802-9952-090558ef396d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:00.501355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c749ce-15b3-4802-9952-090558ef396d service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:00.515054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fce8b3-bd2d-441d-ad81-abd6740a146e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:00.515904 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fce8b3-bd2d-441d-ad81-abd6740a146e service placement] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:00.516292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fce8b3-bd2d-441d-ad81-abd6740a146e service placement] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:00.516660 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fce8b3-bd2d-441d-ad81-abd6740a146e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:00.528622 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-27fce8b3-bd2d-441d-ad81-abd6740a146e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:00.579515 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fce8b3-bd2d-441d-ad81-abd6740a146e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:00.586887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fce8b3-bd2d-441d-ad81-abd6740a146e service placement] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:00.596543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e2535-a55c-4d69-b2d4-4408cd96a7f1 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:00.597496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e2535-a55c-4d69-b2d4-4408cd96a7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:00.598113 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e2535-a55c-4d69-b2d4-4408cd96a7f1 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:00.602261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 880/1776] 10.222.0.60 () {54 vars in 961 bytes} [Wed Sep 28 12:23:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:23:00.602431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 897/1777] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:23:00.609702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb36d34-2bc9-410d-abf0-26f532c4025e None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:00.610102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb36d34-2bc9-410d-abf0-26f532c4025e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:00.610477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb36d34-2bc9-410d-abf0-26f532c4025e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:00.632511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:00.649147 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-aeb36d34-2bc9-410d-abf0-26f532c4025e None None] MFA Rules not processed for user `9a2322b1a76d40e49545f5677e247489`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:00.653157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-aeb36d34-2bc9-410d-abf0-26f532c4025e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:00.681932 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 881/1778] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Sep 28 12:23:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:00.689897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fce8b3-bd2d-441d-ad81-abd6740a146e service placement] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:00.707009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fce8b3-bd2d-441d-ad81-abd6740a146e service placement] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:00.720330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba060eb-135c-4cea-8e80-6559482a3b5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:00.721443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba060eb-135c-4cea-8e80-6559482a3b5d service placement] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:00.721916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba060eb-135c-4cea-8e80-6559482a3b5d service placement] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:00.722342 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba060eb-135c-4cea-8e80-6559482a3b5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:00.750174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba060eb-135c-4cea-8e80-6559482a3b5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:00.760288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba060eb-135c-4cea-8e80-6559482a3b5d service placement] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:00.774096 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 898/1779] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:23:01.309737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b6ca9-4731-41be-9210-cf00a3114ea5 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:01.310049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b6ca9-4731-41be-9210-cf00a3114ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:01.310330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b6ca9-4731-41be-9210-cf00a3114ea5 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:01.312699 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 882/1780] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 12:23:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:23:01.329623 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17bd4f-d6c6-430c-8be8-ea4071cf180d None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:01.330093 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17bd4f-d6c6-430c-8be8-ea4071cf180d None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:01.330165 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17bd4f-d6c6-430c-8be8-ea4071cf180d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:01.371568 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-1e17bd4f-d6c6-430c-8be8-ea4071cf180d None None] MFA Rules not processed for user `9a2322b1a76d40e49545f5677e247489`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:01.373372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-1e17bd4f-d6c6-430c-8be8-ea4071cf180d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:01.393217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 899/1781] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 12:23:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:01.411657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b6ca9-4731-41be-9210-cf00a3114ea5 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:01.413560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-914b6ca9-4731-41be-9210-cf00a3114ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:01.455685 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b6ca9-4731-41be-9210-cf00a3114ea5 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:01.478896 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcb861-1e0f-44eb-b391-d36f43676106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:01.479555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcb861-1e0f-44eb-b391-d36f43676106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:01.479889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcb861-1e0f-44eb-b391-d36f43676106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:01.480162 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcb861-1e0f-44eb-b391-d36f43676106 admin admin] PATH_INFO: `/v3/projects/7ac45189b8fd4821914bd589696ed26d` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:01.485980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcb861-1e0f-44eb-b391-d36f43676106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ac45189b8fd4821914bd589696ed26d)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:01.488074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcb861-1e0f-44eb-b391-d36f43676106 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:01.498529 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17bd4f-d6c6-430c-8be8-ea4071cf180d None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:01.504668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:01.516764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17bd4f-d6c6-430c-8be8-ea4071cf180d None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:01.531979 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364be64-4fcd-4a43-b375-23b2288fbe9f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:01.534089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364be64-4fcd-4a43-b375-23b2288fbe9f service placement] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:01.534550 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364be64-4fcd-4a43-b375-23b2288fbe9f service placement] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:01.534856 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364be64-4fcd-4a43-b375-23b2288fbe9f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:01.559817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364be64-4fcd-4a43-b375-23b2288fbe9f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:01.562818 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364be64-4fcd-4a43-b375-23b2288fbe9f service placement] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:01.590082 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 900/1782] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:23:01.604209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-dafcb861-1e0f-44eb-b391-d36f43676106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ac45189b8fd4821914bd589696ed26d', 'request_id': 'req-dafcb861-1e0f-44eb-b391-d36f43676106'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:01.611836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-dafcb861-1e0f-44eb-b391-d36f43676106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ac45189b8fd4821914bd589696ed26d', 'request_id': 'req-dafcb861-1e0f-44eb-b391-d36f43676106'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:01.624200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-dafcb861-1e0f-44eb-b391-d36f43676106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ac45189b8fd4821914bd589696ed26d', 'request_id': 'req-dafcb861-1e0f-44eb-b391-d36f43676106'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:01.638907 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-dafcb861-1e0f-44eb-b391-d36f43676106 admin admin] The token cache is being invalidate because project 7ac45189b8fd4821914bd589696ed26d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:23:01.639205 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-dafcb861-1e0f-44eb-b391-d36f43676106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:01.639951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-dafcb861-1e0f-44eb-b391-d36f43676106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:01.645962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364be64-4fcd-4a43-b375-23b2288fbe9f service placement] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:01.647524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 883/1783] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:23:01 2022] DELETE /identity/v3/projects/7ac45189b8fd4821914bd589696ed26d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:01.657261 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364be64-4fcd-4a43-b375-23b2288fbe9f service placement] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:01.746749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de73aa-24b2-476e-a3d1-c72e18250829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:01.747522 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de73aa-24b2-476e-a3d1-c72e18250829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:01.747902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de73aa-24b2-476e-a3d1-c72e18250829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:01.748292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de73aa-24b2-476e-a3d1-c72e18250829 admin admin] PATH_INFO: `/v3/users/fb43dfcacb13409a90dd76417f4e07cd` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:01.750363 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de73aa-24b2-476e-a3d1-c72e18250829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb43dfcacb13409a90dd76417f4e07cd)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:01.752672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de73aa-24b2-476e-a3d1-c72e18250829 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:01.909788 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b5de73aa-24b2-476e-a3d1-c72e18250829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb43dfcacb13409a90dd76417f4e07cd'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:01.915003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b5de73aa-24b2-476e-a3d1-c72e18250829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb43dfcacb13409a90dd76417f4e07cd'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:01.928196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b5de73aa-24b2-476e-a3d1-c72e18250829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb43dfcacb13409a90dd76417f4e07cd'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:01.929064 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b5de73aa-24b2-476e-a3d1-c72e18250829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb43dfcacb13409a90dd76417f4e07cd'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:01.947778 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b5de73aa-24b2-476e-a3d1-c72e18250829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb43dfcacb13409a90dd76417f4e07cd'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:01.979745 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 901/1784] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:23:01 2022] DELETE /identity/v3/users/fb43dfcacb13409a90dd76417f4e07cd => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:02.533406 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcb861-1e0f-44eb-b391-d36f43676106 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:02.534968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-dafcb861-1e0f-44eb-b391-d36f43676106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:02.555658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcb861-1e0f-44eb-b391-d36f43676106 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:02.631432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5357e7a-1333-4c55-9062-e1f552dc69ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:02.632109 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5357e7a-1333-4c55-9062-e1f552dc69ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:02.632378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5357e7a-1333-4c55-9062-e1f552dc69ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:02.632892 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5357e7a-1333-4c55-9062-e1f552dc69ee admin admin] PATH_INFO: `/v3/users/ef81663b621e458899ad0eaa324a81fe` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:02.634148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5357e7a-1333-4c55-9062-e1f552dc69ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef81663b621e458899ad0eaa324a81fe)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:02.637438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5357e7a-1333-4c55-9062-e1f552dc69ee admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:02.709628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-a5357e7a-1333-4c55-9062-e1f552dc69ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef81663b621e458899ad0eaa324a81fe'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:02.710361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-a5357e7a-1333-4c55-9062-e1f552dc69ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef81663b621e458899ad0eaa324a81fe'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:02.725874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-a5357e7a-1333-4c55-9062-e1f552dc69ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef81663b621e458899ad0eaa324a81fe'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:02.734208 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-a5357e7a-1333-4c55-9062-e1f552dc69ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef81663b621e458899ad0eaa324a81fe'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:02.735039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-a5357e7a-1333-4c55-9062-e1f552dc69ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef81663b621e458899ad0eaa324a81fe'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:02.735153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de73aa-24b2-476e-a3d1-c72e18250829 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:02.736735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b5de73aa-24b2-476e-a3d1-c72e18250829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:02.752606 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de73aa-24b2-476e-a3d1-c72e18250829 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:02.761693 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 884/1785] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:23:02 2022] DELETE /identity/v3/users/ef81663b621e458899ad0eaa324a81fe => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:02.774330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49d51c-b9ee-41b8-926d-84b8516aeb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:02.774996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49d51c-b9ee-41b8-926d-84b8516aeb26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:02.775274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49d51c-b9ee-41b8-926d-84b8516aeb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:02.775746 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49d51c-b9ee-41b8-926d-84b8516aeb26 admin admin] PATH_INFO: `/v3/projects/879c3e76c79044568d4bb2993e6e8b97` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:02.777353 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49d51c-b9ee-41b8-926d-84b8516aeb26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=879c3e76c79044568d4bb2993e6e8b97)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:02.779737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49d51c-b9ee-41b8-926d-84b8516aeb26 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:02.851239 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5e49d51c-b9ee-41b8-926d-84b8516aeb26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '879c3e76c79044568d4bb2993e6e8b97', 'request_id': 'req-5e49d51c-b9ee-41b8-926d-84b8516aeb26'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:02.859318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5e49d51c-b9ee-41b8-926d-84b8516aeb26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '879c3e76c79044568d4bb2993e6e8b97', 'request_id': 'req-5e49d51c-b9ee-41b8-926d-84b8516aeb26'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:02.874253 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5e49d51c-b9ee-41b8-926d-84b8516aeb26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '879c3e76c79044568d4bb2993e6e8b97', 'request_id': 'req-5e49d51c-b9ee-41b8-926d-84b8516aeb26'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:02.879734 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:02.902053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5e49d51c-b9ee-41b8-926d-84b8516aeb26 admin admin] The token cache is being invalidate because project 879c3e76c79044568d4bb2993e6e8b97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:23:02.903272 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5e49d51c-b9ee-41b8-926d-84b8516aeb26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:02.904510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5e49d51c-b9ee-41b8-926d-84b8516aeb26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:02.916499 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89caf9b-f411-4113-a51e-4e7bfc688572 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:02.917151 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89caf9b-f411-4113-a51e-4e7bfc688572 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:02.917592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89caf9b-f411-4113-a51e-4e7bfc688572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:02.920864 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 902/1786] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:23:02 2022] DELETE /identity/v3/projects/879c3e76c79044568d4bb2993e6e8b97 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:23:02.927814 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:02.946295 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-a89caf9b-f411-4113-a51e-4e7bfc688572 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:02.948571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-a89caf9b-f411-4113-a51e-4e7bfc688572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:03.065872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 885/1787] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:03.076783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49d51c-b9ee-41b8-926d-84b8516aeb26 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:03.078402 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5e49d51c-b9ee-41b8-926d-84b8516aeb26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:03.079831 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332593f0-f2c4-48f6-9c09-11acfa8148b5 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:03.080361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332593f0-f2c4-48f6-9c09-11acfa8148b5 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:03.081448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332593f0-f2c4-48f6-9c09-11acfa8148b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:03.099107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49d51c-b9ee-41b8-926d-84b8516aeb26 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:03.120918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f34deb-f53f-4033-b54f-a7daeb56210d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:03.129937 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f34deb-f53f-4033-b54f-a7daeb56210d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:03.131092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f34deb-f53f-4033-b54f-a7daeb56210d admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:03.131561 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f34deb-f53f-4033-b54f-a7daeb56210d admin admin] PATH_INFO: `/v3/projects/20c8f2f67dce4d119916257372f7db75` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:03.133741 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f34deb-f53f-4033-b54f-a7daeb56210d admin admin] RBAC: Authorizing `identity:delete_project(project_id=20c8f2f67dce4d119916257372f7db75)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:03.135669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-332593f0-f2c4-48f6-9c09-11acfa8148b5 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:03.136238 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f34deb-f53f-4033-b54f-a7daeb56210d admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:03.137122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-332593f0-f2c4-48f6-9c09-11acfa8148b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:03.145808 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 886/1788] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:03.150262 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332593f0-f2c4-48f6-9c09-11acfa8148b5 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:03.164826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332593f0-f2c4-48f6-9c09-11acfa8148b5 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:03.185872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac9745-d13b-438c-989e-43536cbdc4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:03.187445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac9745-d13b-438c-989e-43536cbdc4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:03.187734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac9745-d13b-438c-989e-43536cbdc4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:03.188009 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac9745-d13b-438c-989e-43536cbdc4f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:03.189313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac9745-d13b-438c-989e-43536cbdc4f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:23:03.189620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac9745-d13b-438c-989e-43536cbdc4f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:03.191229 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac9745-d13b-438c-989e-43536cbdc4f1 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:03.208829 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 887/1789] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:23:03.213524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac9745-d13b-438c-989e-43536cbdc4f1 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:03.232477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac9745-d13b-438c-989e-43536cbdc4f1 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:03.238001 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-32f34deb-f53f-4033-b54f-a7daeb56210d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20c8f2f67dce4d119916257372f7db75', 'request_id': 'req-32f34deb-f53f-4033-b54f-a7daeb56210d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:03.248496 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-32f34deb-f53f-4033-b54f-a7daeb56210d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20c8f2f67dce4d119916257372f7db75', 'request_id': 'req-32f34deb-f53f-4033-b54f-a7daeb56210d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:03.258843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-32f34deb-f53f-4033-b54f-a7daeb56210d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20c8f2f67dce4d119916257372f7db75', 'request_id': 'req-32f34deb-f53f-4033-b54f-a7daeb56210d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:03.270469 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-32f34deb-f53f-4033-b54f-a7daeb56210d admin admin] The token cache is being invalidate because project 20c8f2f67dce4d119916257372f7db75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:23:03.270867 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-32f34deb-f53f-4033-b54f-a7daeb56210d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:03.271901 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-32f34deb-f53f-4033-b54f-a7daeb56210d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:03.276240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 903/1790] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:23:03 2022] DELETE /identity/v3/projects/20c8f2f67dce4d119916257372f7db75 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:03.277510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f34deb-f53f-4033-b54f-a7daeb56210d admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:03.278971 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-32f34deb-f53f-4033-b54f-a7daeb56210d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:03.303438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bab1f-d832-491d-b5b7-e3d083cf3552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:03.303972 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bab1f-d832-491d-b5b7-e3d083cf3552 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:03.304323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bab1f-d832-491d-b5b7-e3d083cf3552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:03.304659 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323bab1f-d832-491d-b5b7-e3d083cf3552 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:03.305329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bab1f-d832-491d-b5b7-e3d083cf3552 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:03.307545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323bab1f-d832-491d-b5b7-e3d083cf3552 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:03.341663 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 888/1791] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:23:03 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:23:03.346175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bab1f-d832-491d-b5b7-e3d083cf3552 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:03.348090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-323bab1f-d832-491d-b5b7-e3d083cf3552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:03.361313 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f34deb-f53f-4033-b54f-a7daeb56210d admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:03.422037 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323bab1f-d832-491d-b5b7-e3d083cf3552 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:03.426311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ece0e-3001-436a-a35f-b04296fe5df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:03.426827 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3e3f8-ad50-4191-a5b4-717c923ac821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:03.427737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ece0e-3001-436a-a35f-b04296fe5df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:03.428061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3e3f8-ad50-4191-a5b4-717c923ac821 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:03.428255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ece0e-3001-436a-a35f-b04296fe5df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:03.428255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ece0e-3001-436a-a35f-b04296fe5df6 admin admin] PATH_INFO: `/v3/users/cca22bda316748929f5358d6e1f95fef` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:03.428451 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3e3f8-ad50-4191-a5b4-717c923ac821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:03.428725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe3e3f8-ad50-4191-a5b4-717c923ac821 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:03.429341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3e3f8-ad50-4191-a5b4-717c923ac821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:03.429508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ece0e-3001-436a-a35f-b04296fe5df6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cca22bda316748929f5358d6e1f95fef)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:03.431313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe3e3f8-ad50-4191-a5b4-717c923ac821 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:03.431419 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ece0e-3001-436a-a35f-b04296fe5df6 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:03.453742 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:03.465362 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:03.468468 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 889/1792] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:23:03 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:23:03.476572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3e3f8-ad50-4191-a5b4-717c923ac821 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:03.496236 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe3e3f8-ad50-4191-a5b4-717c923ac821 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:03.501201 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa499bf4-abd4-43e9-a7d8-535922622961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:03.501812 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa499bf4-abd4-43e9-a7d8-535922622961 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:03.502092 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa499bf4-abd4-43e9-a7d8-535922622961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:03.502362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa499bf4-abd4-43e9-a7d8-535922622961 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:03.503008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa499bf4-abd4-43e9-a7d8-535922622961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:03.504868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa499bf4-abd4-43e9-a7d8-535922622961 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:03.510379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-484ece0e-3001-436a-a35f-b04296fe5df6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cca22bda316748929f5358d6e1f95fef'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:03.511231 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-484ece0e-3001-436a-a35f-b04296fe5df6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cca22bda316748929f5358d6e1f95fef'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:03.516853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 890/1793] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:23:03.521468 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa499bf4-abd4-43e9-a7d8-535922622961 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:03.522860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-484ece0e-3001-436a-a35f-b04296fe5df6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cca22bda316748929f5358d6e1f95fef'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:03.523466 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-aa499bf4-abd4-43e9-a7d8-535922622961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:03.523977 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-484ece0e-3001-436a-a35f-b04296fe5df6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cca22bda316748929f5358d6e1f95fef'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:03.533745 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-484ece0e-3001-436a-a35f-b04296fe5df6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cca22bda316748929f5358d6e1f95fef'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:03.538120 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa499bf4-abd4-43e9-a7d8-535922622961 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:03.544570 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0281f-75c7-439f-8e38-9768d2236056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:03.545208 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0281f-75c7-439f-8e38-9768d2236056 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:03.545491 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0281f-75c7-439f-8e38-9768d2236056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:03.545761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0281f-75c7-439f-8e38-9768d2236056 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:03.546394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0281f-75c7-439f-8e38-9768d2236056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:03.548359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0281f-75c7-439f-8e38-9768d2236056 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:03.556071 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 904/1794] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:23:03 2022] DELETE /identity/v3/users/cca22bda316748929f5358d6e1f95fef => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:03.561458 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 891/1795] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:23:03.565272 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ece0e-3001-436a-a35f-b04296fe5df6 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:03.632653 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ece0e-3001-436a-a35f-b04296fe5df6 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:03.637752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1057a-b54c-4ef1-8323-f58bdc4c680b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:03.638604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1057a-b54c-4ef1-8323-f58bdc4c680b admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:03.639369 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1057a-b54c-4ef1-8323-f58bdc4c680b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:03.639369 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1057a-b54c-4ef1-8323-f58bdc4c680b admin admin] PATH_INFO: `/v3/projects/a0fcf65afcaf48869246fcb0dda7ceec/users/964a5de03c3a49e2abd8072e6516ac77/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:03.663086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1057a-b54c-4ef1-8323-f58bdc4c680b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0fcf65afcaf48869246fcb0dda7ceec, user_id=964a5de03c3a49e2abd8072e6516ac77, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:03.665765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1057a-b54c-4ef1-8323-f58bdc4c680b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:03.679148 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 905/1796] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:03 2022] PUT /identity/v3/projects/a0fcf65afcaf48869246fcb0dda7ceec/users/964a5de03c3a49e2abd8072e6516ac77/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:03.688976 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb67aed-627b-4f9a-841b-3d222c4df718 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:03.689460 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb67aed-627b-4f9a-841b-3d222c4df718 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:03.689564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb67aed-627b-4f9a-841b-3d222c4df718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:03.721022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-1fb67aed-627b-4f9a-841b-3d222c4df718 None None] MFA Rules not processed for user `964a5de03c3a49e2abd8072e6516ac77`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:03.722353 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1fb67aed-627b-4f9a-841b-3d222c4df718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:03.754134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1057a-b54c-4ef1-8323-f58bdc4c680b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:03.756990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-66b1057a-b54c-4ef1-8323-f58bdc4c680b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:03.809198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 892/1797] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:03.821296 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb67aed-627b-4f9a-841b-3d222c4df718 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:03.822709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1fb67aed-627b-4f9a-841b-3d222c4df718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:03.839660 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1057a-b54c-4ef1-8323-f58bdc4c680b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:03.844660 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93295987-b540-4e8b-af20-cf761de6fabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:03.845326 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93295987-b540-4e8b-af20-cf761de6fabb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:03.845606 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93295987-b540-4e8b-af20-cf761de6fabb admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:03.845880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93295987-b540-4e8b-af20-cf761de6fabb admin admin] PATH_INFO: `/v3/projects/145957776a4a45c183c7569ecef1b245` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:03.846986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93295987-b540-4e8b-af20-cf761de6fabb admin admin] RBAC: Authorizing `identity:delete_project(project_id=145957776a4a45c183c7569ecef1b245)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:03.849284 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93295987-b540-4e8b-af20-cf761de6fabb admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:03.884768 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb67aed-627b-4f9a-841b-3d222c4df718 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:03.888762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba534940-c040-4c65-932b-412347aa7e5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:03.889420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba534940-c040-4c65-932b-412347aa7e5c service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:03.889687 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba534940-c040-4c65-932b-412347aa7e5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:03.889984 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba534940-c040-4c65-932b-412347aa7e5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:03.891117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:03.899622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba534940-c040-4c65-932b-412347aa7e5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:03.901102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba534940-c040-4c65-932b-412347aa7e5c service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:03.911590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 893/1798] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:23:03.915708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-93295987-b540-4e8b-af20-cf761de6fabb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '145957776a4a45c183c7569ecef1b245', 'request_id': 'req-93295987-b540-4e8b-af20-cf761de6fabb'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:03.926495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-93295987-b540-4e8b-af20-cf761de6fabb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '145957776a4a45c183c7569ecef1b245', 'request_id': 'req-93295987-b540-4e8b-af20-cf761de6fabb'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:03.935032 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba534940-c040-4c65-932b-412347aa7e5c service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:03.936287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-93295987-b540-4e8b-af20-cf761de6fabb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '145957776a4a45c183c7569ecef1b245', 'request_id': 'req-93295987-b540-4e8b-af20-cf761de6fabb'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:03.945773 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba534940-c040-4c65-932b-412347aa7e5c service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:03.947286 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-93295987-b540-4e8b-af20-cf761de6fabb admin admin] The token cache is being invalidate because project 145957776a4a45c183c7569ecef1b245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:23:03.947590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-93295987-b540-4e8b-af20-cf761de6fabb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:03.948234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-93295987-b540-4e8b-af20-cf761de6fabb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:03.949828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 906/1799] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:23:03 2022] DELETE /identity/v3/projects/145957776a4a45c183c7569ecef1b245 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:23:03.954437 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa4127-42fe-4c0e-aca7-c6e055cce8ea None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:03.954719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa4127-42fe-4c0e-aca7-c6e055cce8ea None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:03.955051 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa4127-42fe-4c0e-aca7-c6e055cce8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:03.974877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7efa4127-42fe-4c0e-aca7-c6e055cce8ea None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:03.976079 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7efa4127-42fe-4c0e-aca7-c6e055cce8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:04.007142 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2dfb9b-76b7-422f-9d95-e75edfbdde29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:04.007776 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2dfb9b-76b7-422f-9d95-e75edfbdde29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:04.008052 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2dfb9b-76b7-422f-9d95-e75edfbdde29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:04.008319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2dfb9b-76b7-422f-9d95-e75edfbdde29 admin admin] PATH_INFO: `/v3/users/964a5de03c3a49e2abd8072e6516ac77` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:04.009465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2dfb9b-76b7-422f-9d95-e75edfbdde29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=964a5de03c3a49e2abd8072e6516ac77)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:04.011526 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2dfb9b-76b7-422f-9d95-e75edfbdde29 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:04.022053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:04.035565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 907/1800] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:23:04.045249 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22525b7f-cc61-4b4c-93fe-26b0ab838d3f None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:04.045537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22525b7f-cc61-4b4c-93fe-26b0ab838d3f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:04.045817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22525b7f-cc61-4b4c-93fe-26b0ab838d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:04.064242 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ac2dfb9b-76b7-422f-9d95-e75edfbdde29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '964a5de03c3a49e2abd8072e6516ac77'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:04.064921 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ac2dfb9b-76b7-422f-9d95-e75edfbdde29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '964a5de03c3a49e2abd8072e6516ac77'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:04.067847 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-22525b7f-cc61-4b4c-93fe-26b0ab838d3f None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:04.069047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-22525b7f-cc61-4b4c-93fe-26b0ab838d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:04.077526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 908/1801] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:04.080978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ac2dfb9b-76b7-422f-9d95-e75edfbdde29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '964a5de03c3a49e2abd8072e6516ac77'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:04.081435 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22525b7f-cc61-4b4c-93fe-26b0ab838d3f None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:04.105417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ac2dfb9b-76b7-422f-9d95-e75edfbdde29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '964a5de03c3a49e2abd8072e6516ac77'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:04.105417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ac2dfb9b-76b7-422f-9d95-e75edfbdde29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '964a5de03c3a49e2abd8072e6516ac77'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:04.105601 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22525b7f-cc61-4b4c-93fe-26b0ab838d3f None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:04.105601 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75649ad4-bb49-4e8d-b3b8-7d59ec9f47d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:04.105601 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75649ad4-bb49-4e8d-b3b8-7d59ec9f47d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:04.105601 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75649ad4-bb49-4e8d-b3b8-7d59ec9f47d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:04.105922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75649ad4-bb49-4e8d-b3b8-7d59ec9f47d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:04.105922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75649ad4-bb49-4e8d-b3b8-7d59ec9f47d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:23:04.105922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75649ad4-bb49-4e8d-b3b8-7d59ec9f47d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:04.105922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75649ad4-bb49-4e8d-b3b8-7d59ec9f47d6 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:04.106179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 894/1802] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:23:03 2022] DELETE /identity/v3/users/964a5de03c3a49e2abd8072e6516ac77 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:23:04.109969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90955e-1d28-4d08-9fdc-ce832d609aba None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:04.110266 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90955e-1d28-4d08-9fdc-ce832d609aba None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:04.110553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec90955e-1d28-4d08-9fdc-ce832d609aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:04.115891 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 909/1803] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:23:04.119985 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75649ad4-bb49-4e8d-b3b8-7d59ec9f47d6 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:04.130423 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75649ad4-bb49-4e8d-b3b8-7d59ec9f47d6 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:04.131894 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-ec90955e-1d28-4d08-9fdc-ce832d609aba None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:04.133087 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-ec90955e-1d28-4d08-9fdc-ce832d609aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:04.190406 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96358486-383c-4009-844a-3a7ac6140b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:04.191025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96358486-383c-4009-844a-3a7ac6140b8b admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:04.191308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96358486-383c-4009-844a-3a7ac6140b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:04.191583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96358486-383c-4009-844a-3a7ac6140b8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:04.192211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96358486-383c-4009-844a-3a7ac6140b8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:04.194317 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96358486-383c-4009-844a-3a7ac6140b8b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:04.194798 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 895/1804] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:23:04.204641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90955e-1d28-4d08-9fdc-ce832d609aba None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:04.206107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-ec90955e-1d28-4d08-9fdc-ce832d609aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:04.230679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 910/1805] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:23:04 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:23:04.231639 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96358486-383c-4009-844a-3a7ac6140b8b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:04.257102 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:04.306648 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec90955e-1d28-4d08-9fdc-ce832d609aba None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:04.316356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96358486-383c-4009-844a-3a7ac6140b8b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:04.323633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b610d6-d150-4f9f-8e14-72c4f87fcf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:04.324601 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b610d6-d150-4f9f-8e14-72c4f87fcf5e admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:04.325094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b610d6-d150-4f9f-8e14-72c4f87fcf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:04.325577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b610d6-d150-4f9f-8e14-72c4f87fcf5e admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:04.326652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b610d6-d150-4f9f-8e14-72c4f87fcf5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:04.329259 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b610d6-d150-4f9f-8e14-72c4f87fcf5e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:04.357678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 911/1806] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:23:04 2022] POST /identity/v3/users => generated 282 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:23:04.358575 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:04.361590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b610d6-d150-4f9f-8e14-72c4f87fcf5e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:04.368227 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22632c11-4478-4ae1-8842-82a528d5e710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:04.368928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22632c11-4478-4ae1-8842-82a528d5e710 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:04.369314 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22632c11-4478-4ae1-8842-82a528d5e710 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:04.369655 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22632c11-4478-4ae1-8842-82a528d5e710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:04.375829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b610d6-d150-4f9f-8e14-72c4f87fcf5e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:04.379334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22632c11-4478-4ae1-8842-82a528d5e710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:04.381200 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc2027-48c0-4749-a3af-2db693a6a153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:04.381343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22632c11-4478-4ae1-8842-82a528d5e710 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:04.381764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc2027-48c0-4749-a3af-2db693a6a153 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:04.382055 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc2027-48c0-4749-a3af-2db693a6a153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:04.382326 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc2027-48c0-4749-a3af-2db693a6a153 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:04.382963 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc2027-48c0-4749-a3af-2db693a6a153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:04.384594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc2027-48c0-4749-a3af-2db693a6a153 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:04.390858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 896/1807] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:23:04.394643 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 912/1808] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:23:04.398990 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22632c11-4478-4ae1-8842-82a528d5e710 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:04.408577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22632c11-4478-4ae1-8842-82a528d5e710 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:04.414478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2519d9d-5300-4227-aa4f-e02318bafb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:04.415084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2519d9d-5300-4227-aa4f-e02318bafb98 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:04.415370 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2519d9d-5300-4227-aa4f-e02318bafb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:04.415642 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2519d9d-5300-4227-aa4f-e02318bafb98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:04.416284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2519d9d-5300-4227-aa4f-e02318bafb98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:04.417961 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2519d9d-5300-4227-aa4f-e02318bafb98 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:04.429619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 897/1809] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:23:04.433798 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc2027-48c0-4749-a3af-2db693a6a153 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:04.444551 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc2027-48c0-4749-a3af-2db693a6a153 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:04.449597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e88ca-1df7-4751-8500-73e318ee95b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:04.450259 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e88ca-1df7-4751-8500-73e318ee95b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:04.450539 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e88ca-1df7-4751-8500-73e318ee95b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:04.450814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e88ca-1df7-4751-8500-73e318ee95b5 admin admin] PATH_INFO: `/v3/projects/41ea46946f184e1f89d3dce64eae395d/users/286b9ff52cf94f18bf2e92c1333c3a28/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:04.467196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e88ca-1df7-4751-8500-73e318ee95b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ea46946f184e1f89d3dce64eae395d, user_id=286b9ff52cf94f18bf2e92c1333c3a28, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:04.470007 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e88ca-1df7-4751-8500-73e318ee95b5 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:04.483962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 913/1810] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:04 2022] PUT /identity/v3/projects/41ea46946f184e1f89d3dce64eae395d/users/286b9ff52cf94f18bf2e92c1333c3a28/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:04.495229 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b802a30-d44b-455e-9731-f54f40374b8e None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:04.495526 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b802a30-d44b-455e-9731-f54f40374b8e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:04.495808 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b802a30-d44b-455e-9731-f54f40374b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:04.529506 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-0b802a30-d44b-455e-9731-f54f40374b8e None None] MFA Rules not processed for user `286b9ff52cf94f18bf2e92c1333c3a28`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:04.530836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0b802a30-d44b-455e-9731-f54f40374b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:04.614229 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 898/1811] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:04.618796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e88ca-1df7-4751-8500-73e318ee95b5 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:04.715859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b802a30-d44b-455e-9731-f54f40374b8e None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:04.717567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0b802a30-d44b-455e-9731-f54f40374b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:04.726087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e88ca-1df7-4751-8500-73e318ee95b5 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:04.733663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe9ab7-b1ed-40dc-87c4-5b9a891ac5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:04.734283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe9ab7-b1ed-40dc-87c4-5b9a891ac5d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:04.734565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe9ab7-b1ed-40dc-87c4-5b9a891ac5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:04.734841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe9ab7-b1ed-40dc-87c4-5b9a891ac5d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:04.735573 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe9ab7-b1ed-40dc-87c4-5b9a891ac5d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:04.737981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe9ab7-b1ed-40dc-87c4-5b9a891ac5d8 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:04.744436 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b802a30-d44b-455e-9731-f54f40374b8e None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:04.749487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030c3b2-2308-46ed-aa18-7a92f17e2abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:04.750210 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030c3b2-2308-46ed-aa18-7a92f17e2abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:04.750613 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030c3b2-2308-46ed-aa18-7a92f17e2abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:04.750976 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030c3b2-2308-46ed-aa18-7a92f17e2abd admin admin] PATH_INFO: `/v3/projects/a0fcf65afcaf48869246fcb0dda7ceec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:04.752163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030c3b2-2308-46ed-aa18-7a92f17e2abd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0fcf65afcaf48869246fcb0dda7ceec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:04.754605 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030c3b2-2308-46ed-aa18-7a92f17e2abd admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:04.766552 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 914/1812] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:23:04 2022] POST /identity/v3/projects => generated 363 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:23:04.771504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe9ab7-b1ed-40dc-87c4-5b9a891ac5d8 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:04.810012 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6030c3b2-2308-46ed-aa18-7a92f17e2abd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0fcf65afcaf48869246fcb0dda7ceec', 'request_id': 'req-6030c3b2-2308-46ed-aa18-7a92f17e2abd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:04.816715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6030c3b2-2308-46ed-aa18-7a92f17e2abd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0fcf65afcaf48869246fcb0dda7ceec', 'request_id': 'req-6030c3b2-2308-46ed-aa18-7a92f17e2abd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:04.825476 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6030c3b2-2308-46ed-aa18-7a92f17e2abd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0fcf65afcaf48869246fcb0dda7ceec', 'request_id': 'req-6030c3b2-2308-46ed-aa18-7a92f17e2abd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:04.837329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6030c3b2-2308-46ed-aa18-7a92f17e2abd admin admin] The token cache is being invalidate because project a0fcf65afcaf48869246fcb0dda7ceec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:23:04.837622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6030c3b2-2308-46ed-aa18-7a92f17e2abd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:04.838292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6030c3b2-2308-46ed-aa18-7a92f17e2abd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:04.838599 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe9ab7-b1ed-40dc-87c4-5b9a891ac5d8 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:04.839876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 899/1813] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:23:04 2022] DELETE /identity/v3/projects/a0fcf65afcaf48869246fcb0dda7ceec => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:23:04.843722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:04.845507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fff79-9ead-40a5-b6a0-d6bae70e753a None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:04.845799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fff79-9ead-40a5-b6a0-d6bae70e753a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:04.846086 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fff79-9ead-40a5-b6a0-d6bae70e753a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:04.866420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e03fff79-9ead-40a5-b6a0-d6bae70e753a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:04.867583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e03fff79-9ead-40a5-b6a0-d6bae70e753a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:04.901630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0911c-a73a-4757-8b4e-378b20262cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:04.902140 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0911c-a73a-4757-8b4e-378b20262cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:04.902422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0911c-a73a-4757-8b4e-378b20262cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:04.902696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0911c-a73a-4757-8b4e-378b20262cd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:04.903328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0911c-a73a-4757-8b4e-378b20262cd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:04.905254 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0911c-a73a-4757-8b4e-378b20262cd7 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:04.915165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:04.932281 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 900/1814] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:04.938193 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 915/1815] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:23:04 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:23:04.940299 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0911c-a73a-4757-8b4e-378b20262cd7 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:04.940985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b8c6e-6188-44f9-92b2-c5f08d197efd None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:04.941368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b8c6e-6188-44f9-92b2-c5f08d197efd None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:04.941743 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b8c6e-6188-44f9-92b2-c5f08d197efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:04.944133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d3d0911c-a73a-4757-8b4e-378b20262cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:04.957353 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0911c-a73a-4757-8b4e-378b20262cd7 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:04.966265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-8c4b8c6e-6188-44f9-92b2-c5f08d197efd None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:04.967593 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8c4b8c6e-6188-44f9-92b2-c5f08d197efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:04.976875 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 901/1816] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:23:04.981571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65111e-9247-44d3-a922-d1f5696f2c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:04.981806 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b8c6e-6188-44f9-92b2-c5f08d197efd None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:04.982212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65111e-9247-44d3-a922-d1f5696f2c94 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:04.982497 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65111e-9247-44d3-a922-d1f5696f2c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:04.982771 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65111e-9247-44d3-a922-d1f5696f2c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:04.983421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65111e-9247-44d3-a922-d1f5696f2c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:04.985014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a65111e-9247-44d3-a922-d1f5696f2c94 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:04.992455 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b8c6e-6188-44f9-92b2-c5f08d197efd None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:04.998029 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 916/1817] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:23:05.001824 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65111e-9247-44d3-a922-d1f5696f2c94 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:05.002168 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91263880-3eba-44f6-bb95-fef0206e6538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:05.002959 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91263880-3eba-44f6-bb95-fef0206e6538 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:05.003294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91263880-3eba-44f6-bb95-fef0206e6538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:05.003571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91263880-3eba-44f6-bb95-fef0206e6538 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:05.004265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91263880-3eba-44f6-bb95-fef0206e6538 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:23:05.004580 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91263880-3eba-44f6-bb95-fef0206e6538 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:05.007501 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91263880-3eba-44f6-bb95-fef0206e6538 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:05.013663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65111e-9247-44d3-a922-d1f5696f2c94 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:05.019526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5c4e4-da71-49bb-b1ef-ef329fe6febf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:05.020467 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5c4e4-da71-49bb-b1ef-ef329fe6febf admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:05.020761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5c4e4-da71-49bb-b1ef-ef329fe6febf admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:05.021022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5c4e4-da71-49bb-b1ef-ef329fe6febf admin admin] PATH_INFO: `/v3/projects/e2c4c233125548538973fe8eae166047/users/0d385ba23dbe4af2baefe40312fafa30/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:05.023276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 902/1818] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:23:05.030238 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91263880-3eba-44f6-bb95-fef0206e6538 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:05.042355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5c4e4-da71-49bb-b1ef-ef329fe6febf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2c4c233125548538973fe8eae166047, user_id=0d385ba23dbe4af2baefe40312fafa30, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:05.042789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91263880-3eba-44f6-bb95-fef0206e6538 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:05.047134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5c4e4-da71-49bb-b1ef-ef329fe6febf admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:05.048339 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b686e-2122-4c11-8067-7c33164bc476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:05.049089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b686e-2122-4c11-8067-7c33164bc476 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:05.049405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b686e-2122-4c11-8067-7c33164bc476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:05.049685 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b686e-2122-4c11-8067-7c33164bc476 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:05.050322 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b686e-2122-4c11-8067-7c33164bc476 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:05.054092 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b686e-2122-4c11-8067-7c33164bc476 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:05.070726 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 917/1819] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:05 2022] PUT /identity/v3/projects/e2c4c233125548538973fe8eae166047/users/0d385ba23dbe4af2baefe40312fafa30/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:23:05.074730 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5c4e4-da71-49bb-b1ef-ef329fe6febf admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:05.099371 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 903/1820] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:23:05 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:23:05.101056 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b686e-2122-4c11-8067-7c33164bc476 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:05.201512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5c4e4-da71-49bb-b1ef-ef329fe6febf admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:05.224676 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b686e-2122-4c11-8067-7c33164bc476 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:05.229763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798cf5ea-eade-4205-b21b-65ed488db16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:05.230369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cf5ea-eade-4205-b21b-65ed488db16d admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:05.230652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cf5ea-eade-4205-b21b-65ed488db16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:05.230965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798cf5ea-eade-4205-b21b-65ed488db16d admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:05.231586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798cf5ea-eade-4205-b21b-65ed488db16d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:05.233549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798cf5ea-eade-4205-b21b-65ed488db16d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:05.262302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 904/1821] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:23:05 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:23:05.266714 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798cf5ea-eade-4205-b21b-65ed488db16d admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:05.277418 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c301b86-4471-49d9-91ff-3d2248e3925e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:05.278093 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c301b86-4471-49d9-91ff-3d2248e3925e admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:05.278205 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798cf5ea-eade-4205-b21b-65ed488db16d admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:05.278709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c301b86-4471-49d9-91ff-3d2248e3925e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:05.279163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c301b86-4471-49d9-91ff-3d2248e3925e admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:05.279728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c301b86-4471-49d9-91ff-3d2248e3925e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:05.281527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c301b86-4471-49d9-91ff-3d2248e3925e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:05.284273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94919b3-2929-4a44-a330-69e9a4e6105e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:05.284953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94919b3-2929-4a44-a330-69e9a4e6105e admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:05.285329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94919b3-2929-4a44-a330-69e9a4e6105e admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:05.285681 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94919b3-2929-4a44-a330-69e9a4e6105e admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:05.286762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94919b3-2929-4a44-a330-69e9a4e6105e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:05.289122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94919b3-2929-4a44-a330-69e9a4e6105e admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:05.296930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 918/1822] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:23:05.298060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c301b86-4471-49d9-91ff-3d2248e3925e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:05.304392 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 905/1823] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:23:05.307191 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:05.310366 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94919b3-2929-4a44-a330-69e9a4e6105e admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:05.312147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c301b86-4471-49d9-91ff-3d2248e3925e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:05.319766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ab388-5edc-4e81-9b6b-3c2b367f902c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:05.320435 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ab388-5edc-4e81-9b6b-3c2b367f902c admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:05.321198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ab388-5edc-4e81-9b6b-3c2b367f902c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:05.321497 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ab388-5edc-4e81-9b6b-3c2b367f902c admin admin] PATH_INFO: `/v3/projects/e2c4c233125548538973fe8eae166047/users/0d385ba23dbe4af2baefe40312fafa30/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:05.322917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94919b3-2929-4a44-a330-69e9a4e6105e admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:05.323470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ab388-5edc-4e81-9b6b-3c2b367f902c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2c4c233125548538973fe8eae166047, user_id=0d385ba23dbe4af2baefe40312fafa30, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:05.326140 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ab388-5edc-4e81-9b6b-3c2b367f902c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:05.328781 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2a7ca-2f52-40dc-ba18-73f142d552ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:05.329576 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2a7ca-2f52-40dc-ba18-73f142d552ed admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:05.329863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2a7ca-2f52-40dc-ba18-73f142d552ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:05.330155 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2a7ca-2f52-40dc-ba18-73f142d552ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:05.330797 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2a7ca-2f52-40dc-ba18-73f142d552ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:05.332464 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2a7ca-2f52-40dc-ba18-73f142d552ed admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:05.339791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 919/1824] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:05 2022] PUT /identity/v3/projects/e2c4c233125548538973fe8eae166047/users/0d385ba23dbe4af2baefe40312fafa30/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:05.345411 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 906/1825] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:23:05.351073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ab388-5edc-4e81-9b6b-3c2b367f902c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:05.352387 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7bb9d-82f9-43c5-97cf-ac21c726b9af None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:05.352711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7bb9d-82f9-43c5-97cf-ac21c726b9af None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:05.352988 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7bb9d-82f9-43c5-97cf-ac21c726b9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:05.405093 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-b9a7bb9d-82f9-43c5-97cf-ac21c726b9af None None] MFA Rules not processed for user `0d385ba23dbe4af2baefe40312fafa30`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:05.406756 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b9a7bb9d-82f9-43c5-97cf-ac21c726b9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:05.438446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ab388-5edc-4e81-9b6b-3c2b367f902c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:05.443386 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2a57a-ef4c-432b-9732-6aa8545ee7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:05.444063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2a57a-ef4c-432b-9732-6aa8545ee7a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:05.444340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2a57a-ef4c-432b-9732-6aa8545ee7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:05.444603 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2a57a-ef4c-432b-9732-6aa8545ee7a0 admin admin] PATH_INFO: `/v3/projects/06f981e5f412407e83c8d7372a812c00/users/a20f6baad9834f98967c019ae746b48d/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:05.461600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2a57a-ef4c-432b-9732-6aa8545ee7a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06f981e5f412407e83c8d7372a812c00, user_id=a20f6baad9834f98967c019ae746b48d, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:05.464473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2a57a-ef4c-432b-9732-6aa8545ee7a0 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:05.488050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 920/1826] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:05 2022] PUT /identity/v3/projects/06f981e5f412407e83c8d7372a812c00/users/a20f6baad9834f98967c019ae746b48d/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:05.505204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2560937-2bd3-4e9e-9a2d-23a35c242834 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:05.505650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2560937-2bd3-4e9e-9a2d-23a35c242834 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:05.505998 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2560937-2bd3-4e9e-9a2d-23a35c242834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:05.511400 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:05.526900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 907/1827] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:05.531291 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7bb9d-82f9-43c5-97cf-ac21c726b9af None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:05.533113 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b9a7bb9d-82f9-43c5-97cf-ac21c726b9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:05.550362 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-e2560937-2bd3-4e9e-9a2d-23a35c242834 None None] MFA Rules not processed for user `a20f6baad9834f98967c019ae746b48d`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:05.553254 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e2560937-2bd3-4e9e-9a2d-23a35c242834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:05.595345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7bb9d-82f9-43c5-97cf-ac21c726b9af None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:05.628896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 921/1828] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:23:05.640297 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2560937-2bd3-4e9e-9a2d-23a35c242834 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:05.641846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e2560937-2bd3-4e9e-9a2d-23a35c242834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:05.658107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147a983-9391-48e8-9771-b4e47385252d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:05.658630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147a983-9391-48e8-9771-b4e47385252d service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:05.659001 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147a983-9391-48e8-9771-b4e47385252d service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:05.659412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147a983-9391-48e8-9771-b4e47385252d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:05.707623 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2560937-2bd3-4e9e-9a2d-23a35c242834 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:05.711853 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd5941-234b-4a9f-a379-98500206a7ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:05.712024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd5941-234b-4a9f-a379-98500206a7ef service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:05.712299 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd5941-234b-4a9f-a379-98500206a7ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:05.712750 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd5941-234b-4a9f-a379-98500206a7ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:05.722593 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd5941-234b-4a9f-a379-98500206a7ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:05.725000 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd5941-234b-4a9f-a379-98500206a7ef service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:05.731260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147a983-9391-48e8-9771-b4e47385252d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:05.732778 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147a983-9391-48e8-9771-b4e47385252d service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:05.734479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 922/1829] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:23:05.746501 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 908/1830] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:05 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:23:06.920692 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e3a2c-0262-421d-968b-632d8a710b67 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:06.921472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e3a2c-0262-421d-968b-632d8a710b67 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:06.921472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e3a2c-0262-421d-968b-632d8a710b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:06.950542 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:07.090877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-d14e3a2c-0262-421d-968b-632d8a710b67 None None] MFA Rules not processed for user `bc69cf267bea41bf890bc7551d84ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:07.095874 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d14e3a2c-0262-421d-968b-632d8a710b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:07.244177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 923/1831] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 12:23:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:07.360890 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147a983-9391-48e8-9771-b4e47385252d service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:07.363099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-6147a983-9391-48e8-9771-b4e47385252d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:07.371708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e3a2c-0262-421d-968b-632d8a710b67 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:07.465839 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:07.495226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e3a2c-0262-421d-968b-632d8a710b67 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:07.497498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147a983-9391-48e8-9771-b4e47385252d service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:07.503987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6872c-7916-41f8-b48a-b581f02b464c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975a8d7349994520bcbfb5f75e52aac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:07.504359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d80bf-3991-4f4b-a4d8-20a8ac24bfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:07.505856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6872c-7916-41f8-b48a-b581f02b464c service swift] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:07.506204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6872c-7916-41f8-b48a-b581f02b464c service swift] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:07.506545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6872c-7916-41f8-b48a-b581f02b464c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:07.506951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d80bf-3991-4f4b-a4d8-20a8ac24bfe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:07.507604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d80bf-3991-4f4b-a4d8-20a8ac24bfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:07.509501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d80bf-3991-4f4b-a4d8-20a8ac24bfe9 admin admin] PATH_INFO: `/v3/users/286b9ff52cf94f18bf2e92c1333c3a28` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:07.511093 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d80bf-3991-4f4b-a4d8-20a8ac24bfe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=286b9ff52cf94f18bf2e92c1333c3a28)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:07.516530 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6872c-7916-41f8-b48a-b581f02b464c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:07.517069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d80bf-3991-4f4b-a4d8-20a8ac24bfe9 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:07.518296 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6872c-7916-41f8-b48a-b581f02b464c service swift] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:07.529886 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 909/1832] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 12:23:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 12:23:07.619041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2a2d80bf-3991-4f4b-a4d8-20a8ac24bfe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '286b9ff52cf94f18bf2e92c1333c3a28'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:07.619748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2a2d80bf-3991-4f4b-a4d8-20a8ac24bfe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '286b9ff52cf94f18bf2e92c1333c3a28'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:07.637438 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2a2d80bf-3991-4f4b-a4d8-20a8ac24bfe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '286b9ff52cf94f18bf2e92c1333c3a28'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:07.638232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2a2d80bf-3991-4f4b-a4d8-20a8ac24bfe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '286b9ff52cf94f18bf2e92c1333c3a28'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:07.662107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2a2d80bf-3991-4f4b-a4d8-20a8ac24bfe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '286b9ff52cf94f18bf2e92c1333c3a28'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:07.692846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:07.702250 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 924/1833] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:23:07 2022] DELETE /identity/v3/users/286b9ff52cf94f18bf2e92c1333c3a28 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:23:07.709516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23bd5f-e730-42b3-a898-cf81254733d7 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:07.709516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23bd5f-e730-42b3-a898-cf81254733d7 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:07.709516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23bd5f-e730-42b3-a898-cf81254733d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:07.757696 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-ce23bd5f-e730-42b3-a898-cf81254733d7 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:07.759733 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-ce23bd5f-e730-42b3-a898-cf81254733d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:07.878661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 910/1834] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:07.894556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d80bf-3991-4f4b-a4d8-20a8ac24bfe9 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:07.897332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2a2d80bf-3991-4f4b-a4d8-20a8ac24bfe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:08.011183 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d80bf-3991-4f4b-a4d8-20a8ac24bfe9 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:08.017264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf55d8-9950-40f8-a39d-99600ddf45ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:08.018240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf55d8-9950-40f8-a39d-99600ddf45ee service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:08.018711 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf55d8-9950-40f8-a39d-99600ddf45ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:08.019169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf55d8-9950-40f8-a39d-99600ddf45ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:08.031501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf55d8-9950-40f8-a39d-99600ddf45ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:08.033042 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf55d8-9950-40f8-a39d-99600ddf45ee service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:08.055220 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 925/1835] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:23:08.372778 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23bd5f-e730-42b3-a898-cf81254733d7 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:08.375010 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-ce23bd5f-e730-42b3-a898-cf81254733d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:08.389012 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23bd5f-e730-42b3-a898-cf81254733d7 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:08.393933 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:08.394874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:08.395162 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:08.396129 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4 admin admin] PATH_INFO: `/v3/projects/41ea46946f184e1f89d3dce64eae395d` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:08.397409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41ea46946f184e1f89d3dce64eae395d)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:08.399296 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:08.468240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41ea46946f184e1f89d3dce64eae395d', 'request_id': 'req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:08.474827 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41ea46946f184e1f89d3dce64eae395d', 'request_id': 'req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:08.481827 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41ea46946f184e1f89d3dce64eae395d', 'request_id': 'req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:08.493684 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4 admin admin] The token cache is being invalidate because project 41ea46946f184e1f89d3dce64eae395d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:23:08.494091 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:08.494816 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:08.496407 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 911/1836] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:23:08 2022] DELETE /identity/v3/projects/41ea46946f184e1f89d3dce64eae395d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:08.500340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf55d8-9950-40f8-a39d-99600ddf45ee service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:08.510386 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf55d8-9950-40f8-a39d-99600ddf45ee service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:08.565953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95427f8d-d7c9-4a00-8010-0b1a48d635dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:08.566588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95427f8d-d7c9-4a00-8010-0b1a48d635dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:08.566870 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95427f8d-d7c9-4a00-8010-0b1a48d635dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:08.567663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95427f8d-d7c9-4a00-8010-0b1a48d635dd admin admin] PATH_INFO: `/v3/users/0d385ba23dbe4af2baefe40312fafa30` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:08.568786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95427f8d-d7c9-4a00-8010-0b1a48d635dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d385ba23dbe4af2baefe40312fafa30)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:08.570721 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95427f8d-d7c9-4a00-8010-0b1a48d635dd admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:08.624570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-95427f8d-d7c9-4a00-8010-0b1a48d635dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d385ba23dbe4af2baefe40312fafa30'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:08.624570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-95427f8d-d7c9-4a00-8010-0b1a48d635dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d385ba23dbe4af2baefe40312fafa30'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:08.627736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-95427f8d-d7c9-4a00-8010-0b1a48d635dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d385ba23dbe4af2baefe40312fafa30'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:08.628445 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-95427f8d-d7c9-4a00-8010-0b1a48d635dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d385ba23dbe4af2baefe40312fafa30'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:08.636325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-95427f8d-d7c9-4a00-8010-0b1a48d635dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d385ba23dbe4af2baefe40312fafa30'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:08.654282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 926/1837] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:23:08 2022] DELETE /identity/v3/users/0d385ba23dbe4af2baefe40312fafa30 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:08.946613 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:08.948277 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:08.964108 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e6b7a-5003-4276-a46c-6e54d27dc4e4 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:09.024245 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deecaff-ebde-4e0b-b321-61e55022e901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:09.024879 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deecaff-ebde-4e0b-b321-61e55022e901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:09.025188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deecaff-ebde-4e0b-b321-61e55022e901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:09.025467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deecaff-ebde-4e0b-b321-61e55022e901 admin admin] PATH_INFO: `/v3/projects/e2c4c233125548538973fe8eae166047` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:09.026572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deecaff-ebde-4e0b-b321-61e55022e901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2c4c233125548538973fe8eae166047)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:09.028498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deecaff-ebde-4e0b-b321-61e55022e901 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:09.029927 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:09.077202 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8deecaff-ebde-4e0b-b321-61e55022e901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2c4c233125548538973fe8eae166047', 'request_id': 'req-8deecaff-ebde-4e0b-b321-61e55022e901'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:09.083477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8deecaff-ebde-4e0b-b321-61e55022e901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2c4c233125548538973fe8eae166047', 'request_id': 'req-8deecaff-ebde-4e0b-b321-61e55022e901'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:09.094403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8deecaff-ebde-4e0b-b321-61e55022e901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2c4c233125548538973fe8eae166047', 'request_id': 'req-8deecaff-ebde-4e0b-b321-61e55022e901'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:09.105469 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8deecaff-ebde-4e0b-b321-61e55022e901 admin admin] The token cache is being invalidate because project e2c4c233125548538973fe8eae166047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:23:09.105759 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8deecaff-ebde-4e0b-b321-61e55022e901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:09.106435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8deecaff-ebde-4e0b-b321-61e55022e901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:09.108171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 912/1838] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:23:08 2022] DELETE /identity/v3/projects/e2c4c233125548538973fe8eae166047 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:09.112804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fe477-6da1-41b8-8562-160e32427428 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:09.113333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fe477-6da1-41b8-8562-160e32427428 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:09.113472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fe477-6da1-41b8-8562-160e32427428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:09.141520 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-1d5fe477-6da1-41b8-8562-160e32427428 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:09.142857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-1d5fe477-6da1-41b8-8562-160e32427428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:09.170519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:09.248750 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 927/1839] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:23:09.256985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a538b40-86d2-4652-a521-ecad05073eb5 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:09.257316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a538b40-86d2-4652-a521-ecad05073eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:09.257600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a538b40-86d2-4652-a521-ecad05073eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:09.288602 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-0a538b40-86d2-4652-a521-ecad05073eb5 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:09.298135 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0a538b40-86d2-4652-a521-ecad05073eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:09.309435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 913/1840] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:09.316937 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5fe477-6da1-41b8-8562-160e32427428 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:09.326164 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5fe477-6da1-41b8-8562-160e32427428 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:09.330943 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbbca9-a665-47bf-8966-e60c3c19bff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:09.331555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbbca9-a665-47bf-8966-e60c3c19bff1 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:09.331834 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbbca9-a665-47bf-8966-e60c3c19bff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:09.332107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbbca9-a665-47bf-8966-e60c3c19bff1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:09.332799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbbbca9-a665-47bf-8966-e60c3c19bff1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:23:09.333106 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbbbca9-a665-47bf-8966-e60c3c19bff1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:09.334708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbbbca9-a665-47bf-8966-e60c3c19bff1 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:09.346197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 928/1841] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:23:09.351270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a538b40-86d2-4652-a521-ecad05073eb5 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:09.360646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a538b40-86d2-4652-a521-ecad05073eb5 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:09.366190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d2c81-ba44-48c0-b376-6efce61fac19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:09.366797 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d2c81-ba44-48c0-b376-6efce61fac19 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:09.367083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d2c81-ba44-48c0-b376-6efce61fac19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:09.367387 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d2c81-ba44-48c0-b376-6efce61fac19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:09.367981 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d2c81-ba44-48c0-b376-6efce61fac19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:09.369920 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d2c81-ba44-48c0-b376-6efce61fac19 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:09.396692 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 914/1842] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:23:09 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:23:09.401005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbbca9-a665-47bf-8966-e60c3c19bff1 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:09.411638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbbca9-a665-47bf-8966-e60c3c19bff1 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:09.507258 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcefe74-d455-4410-b788-bb7d8c51c763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:09.508229 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcefe74-d455-4410-b788-bb7d8c51c763 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:09.508620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcefe74-d455-4410-b788-bb7d8c51c763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:09.509043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcefe74-d455-4410-b788-bb7d8c51c763 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:09.510605 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcefe74-d455-4410-b788-bb7d8c51c763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:09.513683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcefe74-d455-4410-b788-bb7d8c51c763 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:09.551918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 929/1843] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:23:09 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:23:09.553563 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d2c81-ba44-48c0-b376-6efce61fac19 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:09.567890 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d2c81-ba44-48c0-b376-6efce61fac19 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:09.573097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13e51a-0227-45ac-a8f9-602062e64793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:09.573914 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13e51a-0227-45ac-a8f9-602062e64793 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:09.574268 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13e51a-0227-45ac-a8f9-602062e64793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:09.574620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac13e51a-0227-45ac-a8f9-602062e64793 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:09.575546 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13e51a-0227-45ac-a8f9-602062e64793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:09.577637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac13e51a-0227-45ac-a8f9-602062e64793 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:09.596021 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 915/1844] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:23:09.596610 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcefe74-d455-4410-b788-bb7d8c51c763 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:09.608223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcefe74-d455-4410-b788-bb7d8c51c763 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:09.612835 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fc13f-0f3f-4876-b103-2def0aeabe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:09.613530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fc13f-0f3f-4876-b103-2def0aeabe5a admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:09.613780 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fc13f-0f3f-4876-b103-2def0aeabe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:09.614409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fc13f-0f3f-4876-b103-2def0aeabe5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:09.615401 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fc13f-0f3f-4876-b103-2def0aeabe5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:09.617001 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fc13f-0f3f-4876-b103-2def0aeabe5a admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:09.630253 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 930/1845] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:23:09.635029 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13e51a-0227-45ac-a8f9-602062e64793 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:09.646123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac13e51a-0227-45ac-a8f9-602062e64793 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:09.651239 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22aa3d-a269-4cc4-9d22-e64dc6faba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:09.651918 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22aa3d-a269-4cc4-9d22-e64dc6faba76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:09.652201 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22aa3d-a269-4cc4-9d22-e64dc6faba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:09.652480 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22aa3d-a269-4cc4-9d22-e64dc6faba76 admin admin] PATH_INFO: `/v3/projects/5d08b246eaff4c468832aaa6e70d4abe/users/8da0fdc0addd41468a89eefdc5ac89d6/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:09.679291 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22aa3d-a269-4cc4-9d22-e64dc6faba76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d08b246eaff4c468832aaa6e70d4abe, user_id=8da0fdc0addd41468a89eefdc5ac89d6, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:09.682092 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22aa3d-a269-4cc4-9d22-e64dc6faba76 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:09.716725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 916/1846] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:09 2022] PUT /identity/v3/projects/5d08b246eaff4c468832aaa6e70d4abe/users/8da0fdc0addd41468a89eefdc5ac89d6/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:23:09.729371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb801ac-09a6-43ba-9deb-0843146f9685 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:09.729651 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb801ac-09a6-43ba-9deb-0843146f9685 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:09.730015 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb801ac-09a6-43ba-9deb-0843146f9685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:09.791245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-fdb801ac-09a6-43ba-9deb-0843146f9685 None None] MFA Rules not processed for user `8da0fdc0addd41468a89eefdc5ac89d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:09.792581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-fdb801ac-09a6-43ba-9deb-0843146f9685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:09.866571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 931/1847] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:09.871901 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22aa3d-a269-4cc4-9d22-e64dc6faba76 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:09.938647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22aa3d-a269-4cc4-9d22-e64dc6faba76 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:09.944167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1fd4bf-5829-417d-93ea-6921dd16b6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:09.944779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fd4bf-5829-417d-93ea-6921dd16b6eb admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:09.945063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fd4bf-5829-417d-93ea-6921dd16b6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:09.945370 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fd4bf-5829-417d-93ea-6921dd16b6eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:09.946012 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fd4bf-5829-417d-93ea-6921dd16b6eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:09.947945 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1fd4bf-5829-417d-93ea-6921dd16b6eb admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:09.976440 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 917/1848] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:23:09 2022] POST /identity/v3/projects => generated 391 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:23:09.980733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb801ac-09a6-43ba-9deb-0843146f9685 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:09.993325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb801ac-09a6-43ba-9deb-0843146f9685 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:10.007749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:10.074894 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd15263-61b8-4930-ae06-a79e8ec0ff8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:10.075538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd15263-61b8-4930-ae06-a79e8ec0ff8f admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:10.075820 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd15263-61b8-4930-ae06-a79e8ec0ff8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:10.076097 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd15263-61b8-4930-ae06-a79e8ec0ff8f admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:10.076747 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd15263-61b8-4930-ae06-a79e8ec0ff8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:10.078772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd15263-61b8-4930-ae06-a79e8ec0ff8f admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:10.105030 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 932/1849] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:23:09 2022] POST /identity/v3/users => generated 302 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:23:10.108743 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1fd4bf-5829-417d-93ea-6921dd16b6eb admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:10.121196 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1fd4bf-5829-417d-93ea-6921dd16b6eb admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:10.125710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff490d-ba24-4c42-9498-55fa73d297ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:10.126315 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff490d-ba24-4c42-9498-55fa73d297ba admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:10.126598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff490d-ba24-4c42-9498-55fa73d297ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:10.126871 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff490d-ba24-4c42-9498-55fa73d297ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:10.127510 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff490d-ba24-4c42-9498-55fa73d297ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:10.129097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff490d-ba24-4c42-9498-55fa73d297ba admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:10.137905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:10.147309 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 918/1850] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:23:10.148490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd15263-61b8-4930-ae06-a79e8ec0ff8f admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:10.159660 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd15263-61b8-4930-ae06-a79e8ec0ff8f admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:10.165054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6dca0-6d7c-4323-a358-f40517b953a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:10.165787 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6dca0-6d7c-4323-a358-f40517b953a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:10.166070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6dca0-6d7c-4323-a358-f40517b953a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:10.166346 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6dca0-6d7c-4323-a358-f40517b953a1 admin admin] PATH_INFO: `/v3/projects/0bdcf2777c2a40b993153d87a3870ae0/users/301bd2838d4c4f389a408a4faa54257d/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:10.182187 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6dca0-6d7c-4323-a358-f40517b953a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bdcf2777c2a40b993153d87a3870ae0, user_id=301bd2838d4c4f389a408a4faa54257d, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:10.184825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6dca0-6d7c-4323-a358-f40517b953a1 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:10.198004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 933/1851] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:10 2022] PUT /identity/v3/projects/0bdcf2777c2a40b993153d87a3870ae0/users/301bd2838d4c4f389a408a4faa54257d/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:10.201924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff490d-ba24-4c42-9498-55fa73d297ba admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:10.211785 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff490d-ba24-4c42-9498-55fa73d297ba admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:10.271601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7d322-c3f8-4f0a-9595-b8a41bda03b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:10.272312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7d322-c3f8-4f0a-9595-b8a41bda03b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:10.272602 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7d322-c3f8-4f0a-9595-b8a41bda03b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:10.272872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7d322-c3f8-4f0a-9595-b8a41bda03b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:10.273552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7d322-c3f8-4f0a-9595-b8a41bda03b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:10.275138 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7d322-c3f8-4f0a-9595-b8a41bda03b3 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:10.289247 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 919/1852] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:23:10.293045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6dca0-6d7c-4323-a358-f40517b953a1 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:10.303483 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6dca0-6d7c-4323-a358-f40517b953a1 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:10.308141 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d34a55-ba0c-4613-bb80-bf5c22f64722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:10.308885 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d34a55-ba0c-4613-bb80-bf5c22f64722 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:10.309205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d34a55-ba0c-4613-bb80-bf5c22f64722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:10.309481 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d34a55-ba0c-4613-bb80-bf5c22f64722 admin admin] PATH_INFO: `/v3/projects/0bdcf2777c2a40b993153d87a3870ae0/users/301bd2838d4c4f389a408a4faa54257d/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:10.311378 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d34a55-ba0c-4613-bb80-bf5c22f64722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bdcf2777c2a40b993153d87a3870ae0, user_id=301bd2838d4c4f389a408a4faa54257d, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:10.314034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d34a55-ba0c-4613-bb80-bf5c22f64722 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:10.325334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 934/1853] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:10 2022] PUT /identity/v3/projects/0bdcf2777c2a40b993153d87a3870ae0/users/301bd2838d4c4f389a408a4faa54257d/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:10.336497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807de188-4822-4c3f-9be5-da7e508c216f None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:10.336788 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807de188-4822-4c3f-9be5-da7e508c216f None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:10.337311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807de188-4822-4c3f-9be5-da7e508c216f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:10.368193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-807de188-4822-4c3f-9be5-da7e508c216f None None] MFA Rules not processed for user `301bd2838d4c4f389a408a4faa54257d`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:10.369483 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-807de188-4822-4c3f-9be5-da7e508c216f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:10.447292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 920/1854] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:10 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:10.458223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d34a55-ba0c-4613-bb80-bf5c22f64722 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:10.460004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-51d34a55-ba0c-4613-bb80-bf5c22f64722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:10.517258 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d34a55-ba0c-4613-bb80-bf5c22f64722 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:10.521004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5686b5-5e95-4935-be58-916effe8b7f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:10.521636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5686b5-5e95-4935-be58-916effe8b7f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:10.521916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5686b5-5e95-4935-be58-916effe8b7f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:10.522204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5686b5-5e95-4935-be58-916effe8b7f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:10.531463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5686b5-5e95-4935-be58-916effe8b7f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:10.532885 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5686b5-5e95-4935-be58-916effe8b7f9 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:10.544072 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 935/1855] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:10 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:23:10.598128 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807de188-4822-4c3f-9be5-da7e508c216f None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:10.599777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-807de188-4822-4c3f-9be5-da7e508c216f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:10.611305 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807de188-4822-4c3f-9be5-da7e508c216f None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:10.615554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774465f-845e-4332-8eb9-2ac032324fb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:10.616143 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774465f-845e-4332-8eb9-2ac032324fb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:10.616420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774465f-845e-4332-8eb9-2ac032324fb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:10.616691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774465f-845e-4332-8eb9-2ac032324fb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:10.675891 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774465f-845e-4332-8eb9-2ac032324fb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:10.678245 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774465f-845e-4332-8eb9-2ac032324fb1 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:10.695333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 921/1856] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:23:12.639035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5686b5-5e95-4935-be58-916effe8b7f9 service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:12.652437 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5686b5-5e95-4935-be58-916effe8b7f9 service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:12.656175 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57543c72-1b10-4423-aba7-2ce434944cee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:12.656786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57543c72-1b10-4423-aba7-2ce434944cee service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:12.657068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57543c72-1b10-4423-aba7-2ce434944cee service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:12.657378 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57543c72-1b10-4423-aba7-2ce434944cee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:12.659146 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-57543c72-1b10-4423-aba7-2ce434944cee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:12.734178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57543c72-1b10-4423-aba7-2ce434944cee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:12.735796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57543c72-1b10-4423-aba7-2ce434944cee service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:12.750863 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 936/1857] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:23:20.686644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=91 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:22.742901 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:22.807886 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774465f-845e-4332-8eb9-2ac032324fb1 service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:22.870720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57543c72-1b10-4423-aba7-2ce434944cee service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:22.873288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-57543c72-1b10-4423-aba7-2ce434944cee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:22.885710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774465f-845e-4332-8eb9-2ac032324fb1 service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:22.886484 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57543c72-1b10-4423-aba7-2ce434944cee service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:22.890622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71967c50-afee-4510-8dc4-9fd824af82a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:22.891387 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71967c50-afee-4510-8dc4-9fd824af82a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:22.891527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131be6f8-c305-432c-a4b0-b2228d6dbcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:22.891796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71967c50-afee-4510-8dc4-9fd824af82a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:22.892086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131be6f8-c305-432c-a4b0-b2228d6dbcf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:22.892191 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71967c50-afee-4510-8dc4-9fd824af82a8 admin admin] PATH_INFO: `/v3/users/8da0fdc0addd41468a89eefdc5ac89d6` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:22.892361 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131be6f8-c305-432c-a4b0-b2228d6dbcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:22.892638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131be6f8-c305-432c-a4b0-b2228d6dbcf0 admin admin] PATH_INFO: `/v3/users/a20f6baad9834f98967c019ae746b48d` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:22.893496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71967c50-afee-4510-8dc4-9fd824af82a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8da0fdc0addd41468a89eefdc5ac89d6)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:22.893864 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131be6f8-c305-432c-a4b0-b2228d6dbcf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a20f6baad9834f98967c019ae746b48d)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:22.896222 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131be6f8-c305-432c-a4b0-b2228d6dbcf0 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:22.896345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71967c50-afee-4510-8dc4-9fd824af82a8 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:22.967659 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-71967c50-afee-4510-8dc4-9fd824af82a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8da0fdc0addd41468a89eefdc5ac89d6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:22.968449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-71967c50-afee-4510-8dc4-9fd824af82a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8da0fdc0addd41468a89eefdc5ac89d6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:22.968585 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-131be6f8-c305-432c-a4b0-b2228d6dbcf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a20f6baad9834f98967c019ae746b48d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:22.969641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-131be6f8-c305-432c-a4b0-b2228d6dbcf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a20f6baad9834f98967c019ae746b48d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:22.979125 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-131be6f8-c305-432c-a4b0-b2228d6dbcf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a20f6baad9834f98967c019ae746b48d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:22.979969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-131be6f8-c305-432c-a4b0-b2228d6dbcf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a20f6baad9834f98967c019ae746b48d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:22.986127 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-71967c50-afee-4510-8dc4-9fd824af82a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8da0fdc0addd41468a89eefdc5ac89d6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:22.987829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-131be6f8-c305-432c-a4b0-b2228d6dbcf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a20f6baad9834f98967c019ae746b48d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:22.994977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-71967c50-afee-4510-8dc4-9fd824af82a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8da0fdc0addd41468a89eefdc5ac89d6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:22.996629 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-71967c50-afee-4510-8dc4-9fd824af82a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8da0fdc0addd41468a89eefdc5ac89d6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:23.010221 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 937/1858] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:23:22 2022] DELETE /identity/v3/users/a20f6baad9834f98967c019ae746b48d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:23.011932 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b9ece-72b0-43e1-b3f3-5c6df5edeae2 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:23.012364 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b9ece-72b0-43e1-b3f3-5c6df5edeae2 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:23.012792 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b9ece-72b0-43e1-b3f3-5c6df5edeae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:23.018617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 922/1859] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:23:22 2022] DELETE /identity/v3/users/8da0fdc0addd41468a89eefdc5ac89d6 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:23.020903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab93794-27cb-4860-bca8-5850577dc406 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:23.021473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab93794-27cb-4860-bca8-5850577dc406 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:23.021934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab93794-27cb-4860-bca8-5850577dc406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:23.056013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-ec5b9ece-72b0-43e1-b3f3-5c6df5edeae2 None None] MFA Rules not processed for user `ba75586d3a7d4278ad6c0c31a1bed3df`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:23.057351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-ec5b9ece-72b0-43e1-b3f3-5c6df5edeae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:23.061518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-8ab93794-27cb-4860-bca8-5850577dc406 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:23.062891 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8ab93794-27cb-4860-bca8-5850577dc406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:23.147123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 923/1860] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:23.149151 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330732f7-2c7c-4ed5-a146-ca602be9f75f None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:23.149750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330732f7-2c7c-4ed5-a146-ca602be9f75f None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:23.150218 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330732f7-2c7c-4ed5-a146-ca602be9f75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:23.198274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 938/1861] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Sep 28 12:23:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:23.200703 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b9ece-72b0-43e1-b3f3-5c6df5edeae2 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:23.205041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-ec5b9ece-72b0-43e1-b3f3-5c6df5edeae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:23.209830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-330732f7-2c7c-4ed5-a146-ca602be9f75f None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:23.213202 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-330732f7-2c7c-4ed5-a146-ca602be9f75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:23.227790 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b9ece-72b0-43e1-b3f3-5c6df5edeae2 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:23.232861 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 924/1862] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:23.235150 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127cf74-f165-487a-9371-66649666a27e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:23.235981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127cf74-f165-487a-9371-66649666a27e service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:23.236268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127cf74-f165-487a-9371-66649666a27e service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:23.237166 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127cf74-f165-487a-9371-66649666a27e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:23.238130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330732f7-2c7c-4ed5-a146-ca602be9f75f None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:23.251415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127cf74-f165-487a-9371-66649666a27e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:23.253104 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127cf74-f165-487a-9371-66649666a27e service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:23.264071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330732f7-2c7c-4ed5-a146-ca602be9f75f None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:23.277269 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0bce2-44be-40f5-96d6-f06f0d897ddd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:23.278378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0bce2-44be-40f5-96d6-f06f0d897ddd service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:23.278818 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0bce2-44be-40f5-96d6-f06f0d897ddd service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:23.279847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0bce2-44be-40f5-96d6-f06f0d897ddd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:23.296354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 939/1863] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:23:23.298410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127cf74-f165-487a-9371-66649666a27e service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:23.299034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0bce2-44be-40f5-96d6-f06f0d897ddd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:23.300774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0bce2-44be-40f5-96d6-f06f0d897ddd service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:23.315812 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:23.320136 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127cf74-f165-487a-9371-66649666a27e service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:23.328686 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 925/1864] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:23:23.336508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15151f6d-5c73-466b-a37d-b70ef078d32a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:23.337242 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15151f6d-5c73-466b-a37d-b70ef078d32a service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:23.337475 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15151f6d-5c73-466b-a37d-b70ef078d32a service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:23.337714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15151f6d-5c73-466b-a37d-b70ef078d32a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:23.360016 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15151f6d-5c73-466b-a37d-b70ef078d32a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:23.361786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15151f6d-5c73-466b-a37d-b70ef078d32a service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:23.377883 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 940/1865] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:23:24.397117 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0bce2-44be-40f5-96d6-f06f0d897ddd service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:24.398655 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-27c0bce2-44be-40f5-96d6-f06f0d897ddd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:24.416210 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0bce2-44be-40f5-96d6-f06f0d897ddd service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:24.422898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd367c-ebaa-4178-b6aa-0b3b2dc6b06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:24.427831 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd367c-ebaa-4178-b6aa-0b3b2dc6b06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:24.428609 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd367c-ebaa-4178-b6aa-0b3b2dc6b06d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:24.428959 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd367c-ebaa-4178-b6aa-0b3b2dc6b06d admin admin] PATH_INFO: `/v3/users/f52bb23291ac4b5db5adee5788c06704` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:24.430899 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd367c-ebaa-4178-b6aa-0b3b2dc6b06d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f52bb23291ac4b5db5adee5788c06704)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:24.433339 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcd367c-ebaa-4178-b6aa-0b3b2dc6b06d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:24.499037 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6dcd367c-ebaa-4178-b6aa-0b3b2dc6b06d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f52bb23291ac4b5db5adee5788c06704'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:24.500418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6dcd367c-ebaa-4178-b6aa-0b3b2dc6b06d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f52bb23291ac4b5db5adee5788c06704'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:24.529900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15151f6d-5c73-466b-a37d-b70ef078d32a service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:24.532680 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-15151f6d-5c73-466b-a37d-b70ef078d32a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:24.537203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6dcd367c-ebaa-4178-b6aa-0b3b2dc6b06d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f52bb23291ac4b5db5adee5788c06704'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:24.549972 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6dcd367c-ebaa-4178-b6aa-0b3b2dc6b06d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f52bb23291ac4b5db5adee5788c06704'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:24.551087 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6dcd367c-ebaa-4178-b6aa-0b3b2dc6b06d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f52bb23291ac4b5db5adee5788c06704'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:24.557771 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15151f6d-5c73-466b-a37d-b70ef078d32a service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:24.572845 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 926/1866] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:23:24 2022] DELETE /identity/v3/users/f52bb23291ac4b5db5adee5788c06704 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:24.577071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f31ec-e792-40e9-931d-9c931d9802e2 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:24.577377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f31ec-e792-40e9-931d-9c931d9802e2 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:24.577655 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f31ec-e792-40e9-931d-9c931d9802e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:24.600115 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ed068f-c715-4938-b9cc-ac9f3e7472a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:24.601064 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed068f-c715-4938-b9cc-ac9f3e7472a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:24.601218 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed068f-c715-4938-b9cc-ac9f3e7472a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:24.601484 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ed068f-c715-4938-b9cc-ac9f3e7472a9 admin admin] PATH_INFO: `/v3/projects/06f981e5f412407e83c8d7372a812c00` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:24.602134 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-b99f31ec-e792-40e9-931d-9c931d9802e2 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:24.603327 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b99f31ec-e792-40e9-931d-9c931d9802e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:24.605458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed068f-c715-4938-b9cc-ac9f3e7472a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06f981e5f412407e83c8d7372a812c00)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:24.607376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ed068f-c715-4938-b9cc-ac9f3e7472a9 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:24.702907 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-30ed068f-c715-4938-b9cc-ac9f3e7472a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06f981e5f412407e83c8d7372a812c00', 'request_id': 'req-30ed068f-c715-4938-b9cc-ac9f3e7472a9'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:24.704338 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:24.718501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-30ed068f-c715-4938-b9cc-ac9f3e7472a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06f981e5f412407e83c8d7372a812c00', 'request_id': 'req-30ed068f-c715-4938-b9cc-ac9f3e7472a9'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:24.732753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-30ed068f-c715-4938-b9cc-ac9f3e7472a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06f981e5f412407e83c8d7372a812c00', 'request_id': 'req-30ed068f-c715-4938-b9cc-ac9f3e7472a9'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:24.766579 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-30ed068f-c715-4938-b9cc-ac9f3e7472a9 admin admin] The token cache is being invalidate because project 06f981e5f412407e83c8d7372a812c00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:23:24.767290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-30ed068f-c715-4938-b9cc-ac9f3e7472a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:24.771152 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-30ed068f-c715-4938-b9cc-ac9f3e7472a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:24.781179 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 941/1867] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:23:24 2022] DELETE /identity/v3/projects/06f981e5f412407e83c8d7372a812c00 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:24.783776 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae9348-9303-417c-8b32-5878f0460c83 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:24.784069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae9348-9303-417c-8b32-5878f0460c83 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:24.784340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae9348-9303-417c-8b32-5878f0460c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:24.790638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 927/1868] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:23:24.814263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-30ae9348-9303-417c-8b32-5878f0460c83 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:24.815523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-30ae9348-9303-417c-8b32-5878f0460c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:24.834772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f31ec-e792-40e9-931d-9c931d9802e2 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:24.836192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b99f31ec-e792-40e9-931d-9c931d9802e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:24.938604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 942/1869] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:24.940174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b312ed8-88ab-4a55-a8aa-a4d84083ea58 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:24.940472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b312ed8-88ab-4a55-a8aa-a4d84083ea58 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:24.940748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b312ed8-88ab-4a55-a8aa-a4d84083ea58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:24.956761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f31ec-e792-40e9-931d-9c931d9802e2 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:24.986955 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-4b312ed8-88ab-4a55-a8aa-a4d84083ea58 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:24.992658 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-4b312ed8-88ab-4a55-a8aa-a4d84083ea58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:25.014633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 943/1870] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:25.018001 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b312ed8-88ab-4a55-a8aa-a4d84083ea58 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:25.020227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-4b312ed8-88ab-4a55-a8aa-a4d84083ea58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:25.044382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:25.057957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b312ed8-88ab-4a55-a8aa-a4d84083ea58 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:25.072484 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767961c5-3f54-4764-978a-6f4158ee6f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:25.073178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767961c5-3f54-4764-978a-6f4158ee6f1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:25.074606 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767961c5-3f54-4764-978a-6f4158ee6f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:25.074892 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767961c5-3f54-4764-978a-6f4158ee6f1e admin admin] PATH_INFO: `/v3/projects/5d08b246eaff4c468832aaa6e70d4abe` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:25.077917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767961c5-3f54-4764-978a-6f4158ee6f1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d08b246eaff4c468832aaa6e70d4abe)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:25.084876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767961c5-3f54-4764-978a-6f4158ee6f1e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:25.105831 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b167e-513a-48f9-a2bf-029804e595eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:25.106464 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b167e-513a-48f9-a2bf-029804e595eb service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:25.108727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b167e-513a-48f9-a2bf-029804e595eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:25.109030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b167e-513a-48f9-a2bf-029804e595eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:25.134284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b167e-513a-48f9-a2bf-029804e595eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:25.135967 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b167e-513a-48f9-a2bf-029804e595eb service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:25.164270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 928/1871] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:23:25.165404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b167e-513a-48f9-a2bf-029804e595eb service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:25.180378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b167e-513a-48f9-a2bf-029804e595eb service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:25.194861 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-767961c5-3f54-4764-978a-6f4158ee6f1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d08b246eaff4c468832aaa6e70d4abe', 'request_id': 'req-767961c5-3f54-4764-978a-6f4158ee6f1e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:25.210445 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-767961c5-3f54-4764-978a-6f4158ee6f1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d08b246eaff4c468832aaa6e70d4abe', 'request_id': 'req-767961c5-3f54-4764-978a-6f4158ee6f1e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:25.229495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-767961c5-3f54-4764-978a-6f4158ee6f1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d08b246eaff4c468832aaa6e70d4abe', 'request_id': 'req-767961c5-3f54-4764-978a-6f4158ee6f1e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:25.251949 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-767961c5-3f54-4764-978a-6f4158ee6f1e admin admin] The token cache is being invalidate because project 5d08b246eaff4c468832aaa6e70d4abe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:23:25.253527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-767961c5-3f54-4764-978a-6f4158ee6f1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:25.257845 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-767961c5-3f54-4764-978a-6f4158ee6f1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:25.262604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 944/1872] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:23:25 2022] DELETE /identity/v3/projects/5d08b246eaff4c468832aaa6e70d4abe => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:25.272640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767961c5-3f54-4764-978a-6f4158ee6f1e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:25.276761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-767961c5-3f54-4764-978a-6f4158ee6f1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:25.314075 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c68be8-133d-4de5-a5c9-9a8dd2672fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:25.314703 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c68be8-133d-4de5-a5c9-9a8dd2672fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:25.314986 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c68be8-133d-4de5-a5c9-9a8dd2672fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:25.315771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c68be8-133d-4de5-a5c9-9a8dd2672fa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:25.316990 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c68be8-133d-4de5-a5c9-9a8dd2672fa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:23:25.318004 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c68be8-133d-4de5-a5c9-9a8dd2672fa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:25.319464 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c68be8-133d-4de5-a5c9-9a8dd2672fa8 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:25.335204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 929/1873] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:23:25.336600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2951e-5072-4c9d-86f0-616e0f082707 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:25.336898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2951e-5072-4c9d-86f0-616e0f082707 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:25.337201 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2951e-5072-4c9d-86f0-616e0f082707 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:25.338540 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 930/1874] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 12:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:23:25.339827 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2951e-5072-4c9d-86f0-616e0f082707 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:25.342028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-29e2951e-5072-4c9d-86f0-616e0f082707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:25.361541 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2951e-5072-4c9d-86f0-616e0f082707 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:25.369740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000bd3e-8931-4cb9-bbff-a4f94d331a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:25.370368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000bd3e-8931-4cb9-bbff-a4f94d331a20 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:25.370704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000bd3e-8931-4cb9-bbff-a4f94d331a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:25.370987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000bd3e-8931-4cb9-bbff-a4f94d331a20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:25.371777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000bd3e-8931-4cb9-bbff-a4f94d331a20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:25.373802 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000bd3e-8931-4cb9-bbff-a4f94d331a20 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:25.392495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:25.399886 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767961c5-3f54-4764-978a-6f4158ee6f1e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:25.409211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8830219-bd6b-496b-b4b0-0a7abcafeab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:25.410493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8830219-bd6b-496b-b4b0-0a7abcafeab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:25.411210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8830219-bd6b-496b-b4b0-0a7abcafeab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:25.414186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8830219-bd6b-496b-b4b0-0a7abcafeab2 admin admin] PATH_INFO: `/v3/users/301bd2838d4c4f389a408a4faa54257d` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:25.415828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8830219-bd6b-496b-b4b0-0a7abcafeab2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=301bd2838d4c4f389a408a4faa54257d)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:25.418526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8830219-bd6b-496b-b4b0-0a7abcafeab2 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:25.424220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 931/1875] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:23:25 2022] POST /identity/v3/projects => generated 375 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:23:25.427289 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd163df-411b-4c5c-8c0d-d0c633eb5fce None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:25.427888 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd163df-411b-4c5c-8c0d-d0c633eb5fce None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:25.428328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd163df-411b-4c5c-8c0d-d0c633eb5fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:25.460996 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-cbd163df-411b-4c5c-8c0d-d0c633eb5fce None None] MFA Rules not processed for user `982b82e8ac424318a934f9149fb91706`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:25.465460 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-cbd163df-411b-4c5c-8c0d-d0c633eb5fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:25.493002 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b8830219-bd6b-496b-b4b0-0a7abcafeab2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '301bd2838d4c4f389a408a4faa54257d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:25.494427 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b8830219-bd6b-496b-b4b0-0a7abcafeab2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '301bd2838d4c4f389a408a4faa54257d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:25.508725 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b8830219-bd6b-496b-b4b0-0a7abcafeab2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '301bd2838d4c4f389a408a4faa54257d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:25.509998 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b8830219-bd6b-496b-b4b0-0a7abcafeab2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '301bd2838d4c4f389a408a4faa54257d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:25.530961 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b8830219-bd6b-496b-b4b0-0a7abcafeab2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '301bd2838d4c4f389a408a4faa54257d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:25.583217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 945/1876] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:23:25 2022] DELETE /identity/v3/users/301bd2838d4c4f389a408a4faa54257d => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:25.583217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8830219-bd6b-496b-b4b0-0a7abcafeab2 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:25.586015 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 932/1877] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 12:23:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:25.586398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b8830219-bd6b-496b-b4b0-0a7abcafeab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:25.682529 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd163df-411b-4c5c-8c0d-d0c633eb5fce None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:25.684132 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-cbd163df-411b-4c5c-8c0d-d0c633eb5fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:25.714977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd163df-411b-4c5c-8c0d-d0c633eb5fce None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:25.743805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8830219-bd6b-496b-b4b0-0a7abcafeab2 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:25.763217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22265930-65f2-4384-87d1-828f5158e029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:25.763858 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22265930-65f2-4384-87d1-828f5158e029 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:25.764177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22265930-65f2-4384-87d1-828f5158e029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:25.764450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22265930-65f2-4384-87d1-828f5158e029 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:25.765207 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22265930-65f2-4384-87d1-828f5158e029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:25.768993 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22265930-65f2-4384-87d1-828f5158e029 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:25.810307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:25.818728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 946/1878] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:23:25 2022] POST /identity/v3/users => generated 288 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:23:25.822605 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22265930-65f2-4384-87d1-828f5158e029 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:25.836784 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22265930-65f2-4384-87d1-828f5158e029 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:25.845412 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bd2f0-ea08-4862-8208-37f1659880c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:25.846782 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bd2f0-ea08-4862-8208-37f1659880c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:25.846782 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bd2f0-ea08-4862-8208-37f1659880c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:25.847014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5bd2f0-ea08-4862-8208-37f1659880c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:25.849106 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5bd2f0-ea08-4862-8208-37f1659880c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:25.852017 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5bd2f0-ea08-4862-8208-37f1659880c7 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:25.868925 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 947/1879] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:23:25.870759 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bd2f0-ea08-4862-8208-37f1659880c7 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:25.895883 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5bd2f0-ea08-4862-8208-37f1659880c7 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:25.904343 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4a2e7-fd6d-4ead-8af3-a30a8711827e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:25.905908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa1caf-fb65-41b1-b264-d13dfc6d41f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:25.907095 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa1caf-fb65-41b1-b264-d13dfc6d41f2 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:25.907856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa1caf-fb65-41b1-b264-d13dfc6d41f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:25.921609 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4a2e7-fd6d-4ead-8af3-a30a8711827e admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:25.922881 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4a2e7-fd6d-4ead-8af3-a30a8711827e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:25.923693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4a2e7-fd6d-4ead-8af3-a30a8711827e admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:25.927840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4a2e7-fd6d-4ead-8af3-a30a8711827e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:25.928664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa1caf-fb65-41b1-b264-d13dfc6d41f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:25.932052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4a2e7-fd6d-4ead-8af3-a30a8711827e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:25.944683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa1caf-fb65-41b1-b264-d13dfc6d41f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:25.946626 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa1caf-fb65-41b1-b264-d13dfc6d41f2 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:25.954605 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 948/1880] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:23:25.959746 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4a2e7-fd6d-4ead-8af3-a30a8711827e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:25.972525 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4a2e7-fd6d-4ead-8af3-a30a8711827e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:25.984516 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d95cbd-4619-47a4-9f72-a0d1600a241d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:25.985228 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d95cbd-4619-47a4-9f72-a0d1600a241d admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:25.985511 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d95cbd-4619-47a4-9f72-a0d1600a241d admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:25.985787 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d95cbd-4619-47a4-9f72-a0d1600a241d admin admin] PATH_INFO: `/v3/projects/6a5863f760bf42a2971567455ecf4a61/users/6a66eb3589d2422ab7ec898f35f54242/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:25.991441 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 933/1881] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:23:26.007776 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa1caf-fb65-41b1-b264-d13dfc6d41f2 service nova] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:26.010893 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-cafa1caf-fb65-41b1-b264-d13dfc6d41f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:26.043863 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d95cbd-4619-47a4-9f72-a0d1600a241d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a5863f760bf42a2971567455ecf4a61, user_id=6a66eb3589d2422ab7ec898f35f54242, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:26.046539 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d95cbd-4619-47a4-9f72-a0d1600a241d admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:26.052157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa1caf-fb65-41b1-b264-d13dfc6d41f2 service nova] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:26.059546 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12865a-6752-4868-a8c8-549a7d4ed7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:26.060197 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12865a-6752-4868-a8c8-549a7d4ed7c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:26.060491 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12865a-6752-4868-a8c8-549a7d4ed7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:26.061417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12865a-6752-4868-a8c8-549a7d4ed7c4 admin admin] PATH_INFO: `/v3/projects/aa5d37fd5ed34d46ab7e58ee52b127a4` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:26.062850 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12865a-6752-4868-a8c8-549a7d4ed7c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa5d37fd5ed34d46ab7e58ee52b127a4)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:26.068486 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12865a-6752-4868-a8c8-549a7d4ed7c4 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:26.075236 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 949/1882] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:25 2022] PUT /identity/v3/projects/6a5863f760bf42a2971567455ecf4a61/users/6a66eb3589d2422ab7ec898f35f54242/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:26.077865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c3e6b-6dd6-475c-8a9d-4981c50a9627 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:26.078166 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c3e6b-6dd6-475c-8a9d-4981c50a9627 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:26.078425 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c3e6b-6dd6-475c-8a9d-4981c50a9627 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:26.082870 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 950/1883] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 12:23:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:23:26.087551 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fb6bb-7c6d-4f66-b09d-7424d30b77bc None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:26.087837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fb6bb-7c6d-4f66-b09d-7424d30b77bc None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:26.088115 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fb6bb-7c6d-4f66-b09d-7424d30b77bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:26.126044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-5c5fb6bb-7c6d-4f66-b09d-7424d30b77bc None None] MFA Rules not processed for user `6a66eb3589d2422ab7ec898f35f54242`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:26.130206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5c5fb6bb-7c6d-4f66-b09d-7424d30b77bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:26.140844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:26.172427 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-bd12865a-6752-4868-a8c8-549a7d4ed7c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa5d37fd5ed34d46ab7e58ee52b127a4', 'request_id': 'req-bd12865a-6752-4868-a8c8-549a7d4ed7c4'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:26.188534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-bd12865a-6752-4868-a8c8-549a7d4ed7c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa5d37fd5ed34d46ab7e58ee52b127a4', 'request_id': 'req-bd12865a-6752-4868-a8c8-549a7d4ed7c4'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:26.202046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-bd12865a-6752-4868-a8c8-549a7d4ed7c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa5d37fd5ed34d46ab7e58ee52b127a4', 'request_id': 'req-bd12865a-6752-4868-a8c8-549a7d4ed7c4'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:26.221360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-bd12865a-6752-4868-a8c8-549a7d4ed7c4 admin admin] The token cache is being invalidate because project aa5d37fd5ed34d46ab7e58ee52b127a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:23:26.221792 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-bd12865a-6752-4868-a8c8-549a7d4ed7c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:26.223374 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-bd12865a-6752-4868-a8c8-549a7d4ed7c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:26.225829 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 934/1884] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:23:25 2022] DELETE /identity/v3/projects/aa5d37fd5ed34d46ab7e58ee52b127a4 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:26.229172 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeaef43-437a-48c6-895c-c7ce374774cf None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:26.229172 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeaef43-437a-48c6-895c-c7ce374774cf None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:26.229172 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeaef43-437a-48c6-895c-c7ce374774cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:26.253239 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-caeaef43-437a-48c6-895c-c7ce374774cf None None] MFA Rules not processed for user `982b82e8ac424318a934f9149fb91706`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:26.255416 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-caeaef43-437a-48c6-895c-c7ce374774cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:26.277950 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 951/1885] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:23:26.279070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fb6bb-7c6d-4f66-b09d-7424d30b77bc None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:26.280998 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5c5fb6bb-7c6d-4f66-b09d-7424d30b77bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:26.372586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 935/1886] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 12:23:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:26.376198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce04b97-41db-42d3-8ca7-2765e39629e4 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:26.376198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce04b97-41db-42d3-8ca7-2765e39629e4 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:26.376198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce04b97-41db-42d3-8ca7-2765e39629e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:26.388387 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fb6bb-7c6d-4f66-b09d-7424d30b77bc None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:26.405588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-5ce04b97-41db-42d3-8ca7-2765e39629e4 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:26.406820 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5ce04b97-41db-42d3-8ca7-2765e39629e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:26.443512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:26.465057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34c930-ed9c-4ff8-8798-2181d7622411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:26.465855 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34c930-ed9c-4ff8-8798-2181d7622411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:26.466133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34c930-ed9c-4ff8-8798-2181d7622411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:26.466427 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34c930-ed9c-4ff8-8798-2181d7622411 admin admin] PATH_INFO: `/v3/projects/0bdcf2777c2a40b993153d87a3870ae0` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:26.467730 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34c930-ed9c-4ff8-8798-2181d7622411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bdcf2777c2a40b993153d87a3870ae0)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:26.469943 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34c930-ed9c-4ff8-8798-2181d7622411 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:26.481652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 936/1887] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:26.482649 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce04b97-41db-42d3-8ca7-2765e39629e4 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:26.484103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5ce04b97-41db-42d3-8ca7-2765e39629e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:26.524895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6f34c930-ed9c-4ff8-8798-2181d7622411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bdcf2777c2a40b993153d87a3870ae0', 'request_id': 'req-6f34c930-ed9c-4ff8-8798-2181d7622411'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:26.531411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6f34c930-ed9c-4ff8-8798-2181d7622411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bdcf2777c2a40b993153d87a3870ae0', 'request_id': 'req-6f34c930-ed9c-4ff8-8798-2181d7622411'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:26.535263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:26.542136 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6f34c930-ed9c-4ff8-8798-2181d7622411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bdcf2777c2a40b993153d87a3870ae0', 'request_id': 'req-6f34c930-ed9c-4ff8-8798-2181d7622411'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:26.554206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6f34c930-ed9c-4ff8-8798-2181d7622411 admin admin] The token cache is being invalidate because project 0bdcf2777c2a40b993153d87a3870ae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:23:26.554497 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6f34c930-ed9c-4ff8-8798-2181d7622411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:26.555290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6f34c930-ed9c-4ff8-8798-2181d7622411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:26.556549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce04b97-41db-42d3-8ca7-2765e39629e4 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:26.556800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 952/1888] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:23:26 2022] DELETE /identity/v3/projects/0bdcf2777c2a40b993153d87a3870ae0 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:26.560264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34c930-ed9c-4ff8-8798-2181d7622411 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:26.560264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-6f34c930-ed9c-4ff8-8798-2181d7622411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:26.607734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb4023-db41-4770-9b9a-0a17642a325a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:26.608349 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb4023-db41-4770-9b9a-0a17642a325a service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:26.608631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb4023-db41-4770-9b9a-0a17642a325a service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:26.608904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb4023-db41-4770-9b9a-0a17642a325a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:26.610665 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b7bb4023-db41-4770-9b9a-0a17642a325a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:26.637719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34c930-ed9c-4ff8-8798-2181d7622411 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:26.645311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5c68c-b3e6-4cff-8e31-afeb4facc5ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:26.645951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5c68c-b3e6-4cff-8e31-afeb4facc5ac service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:26.646234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5c68c-b3e6-4cff-8e31-afeb4facc5ac service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:26.646504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5c68c-b3e6-4cff-8e31-afeb4facc5ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:26.648243 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e6c5c68c-b3e6-4cff-8e31-afeb4facc5ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:26.660756 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5c68c-b3e6-4cff-8e31-afeb4facc5ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:26.662379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5c68c-b3e6-4cff-8e31-afeb4facc5ac service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:26.673042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb4023-db41-4770-9b9a-0a17642a325a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:26.674225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 953/1889] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:23:26.674551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb4023-db41-4770-9b9a-0a17642a325a service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:26.675327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a0ec8-2657-4f9e-97f9-6535bc9c9fbe None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:26.675699 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a0ec8-2657-4f9e-97f9-6535bc9c9fbe None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:26.675896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a0ec8-2657-4f9e-97f9-6535bc9c9fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:26.686790 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 937/1890] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:23:26.688036 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b553a6-df32-4287-842a-a8d5db255d62 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:26.688325 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b553a6-df32-4287-842a-a8d5db255d62 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:26.688598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b553a6-df32-4287-842a-a8d5db255d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:26.708491 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-054a0ec8-2657-4f9e-97f9-6535bc9c9fbe None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:26.709735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-054a0ec8-2657-4f9e-97f9-6535bc9c9fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:26.730454 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-56b553a6-df32-4287-842a-a8d5db255d62 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:26.731667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-56b553a6-df32-4287-842a-a8d5db255d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:26.795999 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:26.807507 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 954/1891] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:26.811858 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a0ec8-2657-4f9e-97f9-6535bc9c9fbe None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:26.821572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 938/1892] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:26.829152 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a0ec8-2657-4f9e-97f9-6535bc9c9fbe None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:26.829152 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f435cb-8063-4f53-bf2e-edf5745b456e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:26.829794 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f435cb-8063-4f53-bf2e-edf5745b456e admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:26.830595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f435cb-8063-4f53-bf2e-edf5745b456e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:26.830874 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f435cb-8063-4f53-bf2e-edf5745b456e admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:26.831837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f435cb-8063-4f53-bf2e-edf5745b456e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:23:26.832555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece776c-e021-4db6-8333-74ede4c2c1a6 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:26.832801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f435cb-8063-4f53-bf2e-edf5745b456e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:26.832946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece776c-e021-4db6-8333-74ede4c2c1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:26.833065 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece776c-e021-4db6-8333-74ede4c2c1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:26.834649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f435cb-8063-4f53-bf2e-edf5745b456e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:26.854477 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 955/1893] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:23:26.859354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f435cb-8063-4f53-bf2e-edf5745b456e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:26.866625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-9ece776c-e021-4db6-8333-74ede4c2c1a6 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:26.868394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-9ece776c-e021-4db6-8333-74ede4c2c1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:26.887791 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 939/1894] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:26.887932 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f435cb-8063-4f53-bf2e-edf5745b456e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:26.892172 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece776c-e021-4db6-8333-74ede4c2c1a6 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:26.894865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb17590-99cb-4e9e-9a6a-07805f7b8ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:26.895577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb17590-99cb-4e9e-9a6a-07805f7b8ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:26.895856 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb17590-99cb-4e9e-9a6a-07805f7b8ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:26.896130 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb17590-99cb-4e9e-9a6a-07805f7b8ee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:26.896886 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb17590-99cb-4e9e-9a6a-07805f7b8ee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:26.898815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb17590-99cb-4e9e-9a6a-07805f7b8ee6 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:26.903356 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece776c-e021-4db6-8333-74ede4c2c1a6 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:26.907421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653bc67-5116-4b22-aff7-a1b7dfc1f2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:26.908104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653bc67-5116-4b22-aff7-a1b7dfc1f2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:26.908478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653bc67-5116-4b22-aff7-a1b7dfc1f2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:26.908822 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653bc67-5116-4b22-aff7-a1b7dfc1f2d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:26.909573 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653bc67-5116-4b22-aff7-a1b7dfc1f2d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:23:26.910726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653bc67-5116-4b22-aff7-a1b7dfc1f2d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:26.912743 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653bc67-5116-4b22-aff7-a1b7dfc1f2d5 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:26.925179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 940/1895] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:23:26.931994 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 956/1896] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:23:26 2022] POST /identity/v3/projects => generated 357 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:23:26.933047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653bc67-5116-4b22-aff7-a1b7dfc1f2d5 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:26.933239 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb17590-99cb-4e9e-9a6a-07805f7b8ee6 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:26.944993 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653bc67-5116-4b22-aff7-a1b7dfc1f2d5 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:26.984354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:27.015779 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb17590-99cb-4e9e-9a6a-07805f7b8ee6 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.018739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd9676-30ab-4452-8a1a-bc0152328fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.019590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd9676-30ab-4452-8a1a-bc0152328fba admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.020084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd9676-30ab-4452-8a1a-bc0152328fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.020368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd9676-30ab-4452-8a1a-bc0152328fba admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.020988 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966de3a-9d07-40af-ae43-5df98d47e390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.021122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd9676-30ab-4452-8a1a-bc0152328fba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.021644 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966de3a-9d07-40af-ae43-5df98d47e390 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.021933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966de3a-9d07-40af-ae43-5df98d47e390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.022261 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966de3a-9d07-40af-ae43-5df98d47e390 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.022820 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966de3a-9d07-40af-ae43-5df98d47e390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.023182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd9676-30ab-4452-8a1a-bc0152328fba admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.025008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966de3a-9d07-40af-ae43-5df98d47e390 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.049302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 941/1897] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:23:26 2022] POST /identity/v3/users => generated 279 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:23:27.054181 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd9676-30ab-4452-8a1a-bc0152328fba admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.062856 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 957/1898] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:23:26 2022] POST /identity/v3/projects => generated 367 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:23:27.065929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd9676-30ab-4452-8a1a-bc0152328fba admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.074567 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966de3a-9d07-40af-ae43-5df98d47e390 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.178207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16fe11-0064-4bd2-bf6c-6df820ccd47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.181162 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16fe11-0064-4bd2-bf6c-6df820ccd47d admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.181162 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16fe11-0064-4bd2-bf6c-6df820ccd47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.181162 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16fe11-0064-4bd2-bf6c-6df820ccd47d admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.181162 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16fe11-0064-4bd2-bf6c-6df820ccd47d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.182465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16fe11-0064-4bd2-bf6c-6df820ccd47d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.189472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966de3a-9d07-40af-ae43-5df98d47e390 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.203308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1208b-438b-42eb-8a66-9c4d59c878bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.203455 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 942/1899] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:23:27.203671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1208b-438b-42eb-8a66-9c4d59c878bc admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.203968 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1208b-438b-42eb-8a66-9c4d59c878bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.204245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1208b-438b-42eb-8a66-9c4d59c878bc admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.204832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16fe11-0064-4bd2-bf6c-6df820ccd47d admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.204931 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1208b-438b-42eb-8a66-9c4d59c878bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.207727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1208b-438b-42eb-8a66-9c4d59c878bc admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.215474 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16fe11-0064-4bd2-bf6c-6df820ccd47d admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.237504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 958/1900] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:23:27 2022] POST /identity/v3/users => generated 284 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:23:27.242169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1208b-438b-42eb-8a66-9c4d59c878bc admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.247435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db3afb-0abd-4302-b92d-54209a034a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.248307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db3afb-0abd-4302-b92d-54209a034a9e admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.248600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db3afb-0abd-4302-b92d-54209a034a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.251239 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db3afb-0abd-4302-b92d-54209a034a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.251948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db3afb-0abd-4302-b92d-54209a034a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.253634 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db3afb-0abd-4302-b92d-54209a034a9e admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.256144 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1208b-438b-42eb-8a66-9c4d59c878bc admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.262964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4cdff-0ab0-43c6-a609-c9f0ed01da66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.263832 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4cdff-0ab0-43c6-a609-c9f0ed01da66 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.264177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4cdff-0ab0-43c6-a609-c9f0ed01da66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.264422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4cdff-0ab0-43c6-a609-c9f0ed01da66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.265188 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4cdff-0ab0-43c6-a609-c9f0ed01da66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.266869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4cdff-0ab0-43c6-a609-c9f0ed01da66 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.278346 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 943/1901] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:23:27.281160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 959/1902] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:23:27.282744 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db3afb-0abd-4302-b92d-54209a034a9e admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.283614 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4cdff-0ab0-43c6-a609-c9f0ed01da66 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.294154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db3afb-0abd-4302-b92d-54209a034a9e admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.294342 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4cdff-0ab0-43c6-a609-c9f0ed01da66 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.298904 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26c13a-de1a-4322-a4c9-4894a43edafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.299046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbbf796-a7e2-4443-98b2-02977ba04fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.299543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26c13a-de1a-4322-a4c9-4894a43edafd admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.299773 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26c13a-de1a-4322-a4c9-4894a43edafd admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.299890 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbf796-a7e2-4443-98b2-02977ba04fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.300046 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26c13a-de1a-4322-a4c9-4894a43edafd admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.300230 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbf796-a7e2-4443-98b2-02977ba04fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.300602 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbf796-a7e2-4443-98b2-02977ba04fc9 admin admin] PATH_INFO: `/v3/projects/0dc9b3c68d774159a5acb1b15837da13/users/680e2b80bbdd4b319eb4bb5faa1fb544/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.301277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c26c13a-de1a-4322-a4c9-4894a43edafd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.303185 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c26c13a-de1a-4322-a4c9-4894a43edafd admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.314602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 960/1903] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:23:27.318896 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbbf796-a7e2-4443-98b2-02977ba04fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dc9b3c68d774159a5acb1b15837da13, user_id=680e2b80bbdd4b319eb4bb5faa1fb544, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.319029 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26c13a-de1a-4322-a4c9-4894a43edafd admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.321726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbbf796-a7e2-4443-98b2-02977ba04fc9 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.329745 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26c13a-de1a-4322-a4c9-4894a43edafd admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.335358 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 944/1904] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:27 2022] PUT /identity/v3/projects/0dc9b3c68d774159a5acb1b15837da13/users/680e2b80bbdd4b319eb4bb5faa1fb544/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:23:27.347417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7651d-9cce-43de-83aa-fa8367b35604 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.347417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7651d-9cce-43de-83aa-fa8367b35604 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.348202 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7651d-9cce-43de-83aa-fa8367b35604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.386946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-c9b7651d-9cce-43de-83aa-fa8367b35604 None None] MFA Rules not processed for user `680e2b80bbdd4b319eb4bb5faa1fb544`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:27.388336 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c9b7651d-9cce-43de-83aa-fa8367b35604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:27.392582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0d40f-7329-45b8-a81f-8ab88426f465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.393360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0d40f-7329-45b8-a81f-8ab88426f465 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.393854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0d40f-7329-45b8-a81f-8ab88426f465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.394117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0d40f-7329-45b8-a81f-8ab88426f465 admin admin] PATH_INFO: `/v3/projects/ec1f3717b7b04b328256ef25c662eab0/users/95ad41337bad4a89bd77922bf4b52b93/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.394300 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:27.414588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0d40f-7329-45b8-a81f-8ab88426f465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec1f3717b7b04b328256ef25c662eab0, user_id=95ad41337bad4a89bd77922bf4b52b93, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.417423 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0d40f-7329-45b8-a81f-8ab88426f465 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.443415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 961/1905] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:27 2022] PUT /identity/v3/projects/ec1f3717b7b04b328256ef25c662eab0/users/95ad41337bad4a89bd77922bf4b52b93/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:23:27.453337 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74143a7-c36d-40d8-b8d3-0c0ae77b298d None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.453621 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74143a7-c36d-40d8-b8d3-0c0ae77b298d None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.453893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74143a7-c36d-40d8-b8d3-0c0ae77b298d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.462578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 945/1906] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:27 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:23:27.466990 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7651d-9cce-43de-83aa-fa8367b35604 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.476533 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:27.478528 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7651d-9cce-43de-83aa-fa8367b35604 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.485086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-f74143a7-c36d-40d8-b8d3-0c0ae77b298d None None] MFA Rules not processed for user `95ad41337bad4a89bd77922bf4b52b93`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:27.486718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-f74143a7-c36d-40d8-b8d3-0c0ae77b298d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:27.539617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c912063-3d22-4882-9d9e-a1b73b880286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.540235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c912063-3d22-4882-9d9e-a1b73b880286 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.540521 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c912063-3d22-4882-9d9e-a1b73b880286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.540794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c912063-3d22-4882-9d9e-a1b73b880286 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.541462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c912063-3d22-4882-9d9e-a1b73b880286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.543555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c912063-3d22-4882-9d9e-a1b73b880286 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.554004 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 962/1907] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:23:27.558511 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74143a7-c36d-40d8-b8d3-0c0ae77b298d None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.570677 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74143a7-c36d-40d8-b8d3-0c0ae77b298d None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.576688 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 946/1908] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:23:27 2022] POST /identity/v3/projects => generated 357 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:23:27.578892 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b5a3b-3307-497d-9c57-f75a8d117667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.579341 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b5a3b-3307-497d-9c57-f75a8d117667 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.579619 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b5a3b-3307-497d-9c57-f75a8d117667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.579913 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b5a3b-3307-497d-9c57-f75a8d117667 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.580543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b5a3b-3307-497d-9c57-f75a8d117667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.582205 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c912063-3d22-4882-9d9e-a1b73b880286 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.582554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b5a3b-3307-497d-9c57-f75a8d117667 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.614167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 963/1909] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:23:27 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:23:27.617879 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b5a3b-3307-497d-9c57-f75a8d117667 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.652390 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c912063-3d22-4882-9d9e-a1b73b880286 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.681055 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b5a3b-3307-497d-9c57-f75a8d117667 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.685777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0624bd0-2ca8-4b21-83a9-12ed2a6b2d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.686391 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0624bd0-2ca8-4b21-83a9-12ed2a6b2d7c admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.686670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0624bd0-2ca8-4b21-83a9-12ed2a6b2d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.686973 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0624bd0-2ca8-4b21-83a9-12ed2a6b2d7c admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.687602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0624bd0-2ca8-4b21-83a9-12ed2a6b2d7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.689519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0624bd0-2ca8-4b21-83a9-12ed2a6b2d7c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.700076 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:27.717617 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 964/1910] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:23:27 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:23:27.721585 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0624bd0-2ca8-4b21-83a9-12ed2a6b2d7c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.723903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28655b00-9848-4651-a490-e7f6db8c2a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.724518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28655b00-9848-4651-a490-e7f6db8c2a24 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.724799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28655b00-9848-4651-a490-e7f6db8c2a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.725071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28655b00-9848-4651-a490-e7f6db8c2a24 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.725760 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28655b00-9848-4651-a490-e7f6db8c2a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.727668 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28655b00-9848-4651-a490-e7f6db8c2a24 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.731781 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0624bd0-2ca8-4b21-83a9-12ed2a6b2d7c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.736307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a79ea3-181f-4dcd-8faa-64e8886ec076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.736933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a79ea3-181f-4dcd-8faa-64e8886ec076 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.737242 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a79ea3-181f-4dcd-8faa-64e8886ec076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.737791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a79ea3-181f-4dcd-8faa-64e8886ec076 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.738693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a79ea3-181f-4dcd-8faa-64e8886ec076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.740265 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a79ea3-181f-4dcd-8faa-64e8886ec076 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.750753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 965/1912] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:23:27.752927 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 947/1911] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:23:27 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:23:27.754564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a79ea3-181f-4dcd-8faa-64e8886ec076 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.755763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28655b00-9848-4651-a490-e7f6db8c2a24 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.764367 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a79ea3-181f-4dcd-8faa-64e8886ec076 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.765702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28655b00-9848-4651-a490-e7f6db8c2a24 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.768958 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a88665-1f08-43ab-b3d4-f5d54f721a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.769636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a88665-1f08-43ab-b3d4-f5d54f721a0e admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.769908 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a88665-1f08-43ab-b3d4-f5d54f721a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.770194 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a88665-1f08-43ab-b3d4-f5d54f721a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.770713 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d25fc-30d0-4a61-98c4-4445bd177c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.770835 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a88665-1f08-43ab-b3d4-f5d54f721a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.771409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d25fc-30d0-4a61-98c4-4445bd177c67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.771720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d25fc-30d0-4a61-98c4-4445bd177c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.771996 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d25fc-30d0-4a61-98c4-4445bd177c67 admin admin] PATH_INFO: `/v3/projects/4d8c0ae0fd7145cf9cd54b43423be2c7/users/7559d138e2f04fdb92e44e77cd4c9c4a/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.772439 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a88665-1f08-43ab-b3d4-f5d54f721a0e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.786427 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 966/1913] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:23:27.790736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a88665-1f08-43ab-b3d4-f5d54f721a0e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.794227 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d25fc-30d0-4a61-98c4-4445bd177c67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d8c0ae0fd7145cf9cd54b43423be2c7, user_id=7559d138e2f04fdb92e44e77cd4c9c4a, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.796928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d25fc-30d0-4a61-98c4-4445bd177c67 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.801447 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a88665-1f08-43ab-b3d4-f5d54f721a0e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.807852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92325b06-1d42-420e-b133-dddf81240ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.808459 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92325b06-1d42-420e-b133-dddf81240ba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.808745 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92325b06-1d42-420e-b133-dddf81240ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.809022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92325b06-1d42-420e-b133-dddf81240ba1 admin admin] PATH_INFO: `/v3/projects/0b143975e7064ff5b962956bc1014668/users/8ef090fb80104395916889a91c717102/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.813626 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 948/1914] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:27 2022] PUT /identity/v3/projects/4d8c0ae0fd7145cf9cd54b43423be2c7/users/7559d138e2f04fdb92e44e77cd4c9c4a/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:23:27.818225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d25fc-30d0-4a61-98c4-4445bd177c67 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.826330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92325b06-1d42-420e-b133-dddf81240ba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b143975e7064ff5b962956bc1014668, user_id=8ef090fb80104395916889a91c717102, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.829059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92325b06-1d42-420e-b133-dddf81240ba1 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.843934 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 967/1915] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:27 2022] PUT /identity/v3/projects/0b143975e7064ff5b962956bc1014668/users/8ef090fb80104395916889a91c717102/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:23:27.848181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92325b06-1d42-420e-b133-dddf81240ba1 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.880597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:27.889553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d25fc-30d0-4a61-98c4-4445bd177c67 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.927239 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92325b06-1d42-420e-b133-dddf81240ba1 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.932258 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fd67f-916e-4762-a08d-a9c8feda2b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.932943 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fd67f-916e-4762-a08d-a9c8feda2b51 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.933311 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fd67f-916e-4762-a08d-a9c8feda2b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.933589 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fd67f-916e-4762-a08d-a9c8feda2b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.934228 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fd67f-916e-4762-a08d-a9c8feda2b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.935805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fd67f-916e-4762-a08d-a9c8feda2b51 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.949298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 968/1916] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:23:27.951068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fd67f-916e-4762-a08d-a9c8feda2b51 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.958766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22916eb-03fc-4529-8693-68e29a4ad4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.959420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22916eb-03fc-4529-8693-68e29a4ad4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.959718 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22916eb-03fc-4529-8693-68e29a4ad4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.959977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22916eb-03fc-4529-8693-68e29a4ad4e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.960618 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22916eb-03fc-4529-8693-68e29a4ad4e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.962335 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22916eb-03fc-4529-8693-68e29a4ad4e1 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.964032 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fd67f-916e-4762-a08d-a9c8feda2b51 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.969283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311c463-e97d-4800-a78e-3ab8a21475ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:27.970257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311c463-e97d-4800-a78e-3ab8a21475ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:27.970903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311c463-e97d-4800-a78e-3ab8a21475ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:27.971398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311c463-e97d-4800-a78e-3ab8a21475ee admin admin] PATH_INFO: `/v3/projects/0b143975e7064ff5b962956bc1014668/users/8ef090fb80104395916889a91c717102/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:27.976271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 949/1917] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:23:27.977607 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22916eb-03fc-4529-8693-68e29a4ad4e1 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:27.978554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311c463-e97d-4800-a78e-3ab8a21475ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b143975e7064ff5b962956bc1014668, user_id=8ef090fb80104395916889a91c717102, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:27.981936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311c463-e97d-4800-a78e-3ab8a21475ee admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:27.990955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22916eb-03fc-4529-8693-68e29a4ad4e1 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:27.996901 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 969/1918] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:27 2022] PUT /identity/v3/projects/0b143975e7064ff5b962956bc1014668/users/8ef090fb80104395916889a91c717102/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:28.007195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c5037-73e1-4c41-b783-a91ded90d2b6 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:28.007733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c5037-73e1-4c41-b783-a91ded90d2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:28.008800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c5037-73e1-4c41-b783-a91ded90d2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:28.051946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:28.061748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7a4c5037-73e1-4c41-b783-a91ded90d2b6 None None] MFA Rules not processed for user `8ef090fb80104395916889a91c717102`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:28.063896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7a4c5037-73e1-4c41-b783-a91ded90d2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:28.070988 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258412c2-a0d0-49a2-892d-f98d2617ad92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:28.072481 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258412c2-a0d0-49a2-892d-f98d2617ad92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:28.072925 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258412c2-a0d0-49a2-892d-f98d2617ad92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:28.073941 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258412c2-a0d0-49a2-892d-f98d2617ad92 admin admin] PATH_INFO: `/v3/projects/4d8c0ae0fd7145cf9cd54b43423be2c7/users/7559d138e2f04fdb92e44e77cd4c9c4a/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:28.076644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258412c2-a0d0-49a2-892d-f98d2617ad92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d8c0ae0fd7145cf9cd54b43423be2c7, user_id=7559d138e2f04fdb92e44e77cd4c9c4a, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:28.079648 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258412c2-a0d0-49a2-892d-f98d2617ad92 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:28.103375 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 950/1919] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:27 2022] PUT /identity/v3/projects/4d8c0ae0fd7145cf9cd54b43423be2c7/users/7559d138e2f04fdb92e44e77cd4c9c4a/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:28.105813 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04466ec6-0144-4269-a17a-70241fcb1b91 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:28.106124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04466ec6-0144-4269-a17a-70241fcb1b91 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:28.106411 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04466ec6-0144-4269-a17a-70241fcb1b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:28.143504 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-04466ec6-0144-4269-a17a-70241fcb1b91 None None] MFA Rules not processed for user `7559d138e2f04fdb92e44e77cd4c9c4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:28.145176 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-04466ec6-0144-4269-a17a-70241fcb1b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:28.166618 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 970/1920] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:28 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:23:28.175288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c5037-73e1-4c41-b783-a91ded90d2b6 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:28.176839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7a4c5037-73e1-4c41-b783-a91ded90d2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:28.288268 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 951/1921] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:23:28.317019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04466ec6-0144-4269-a17a-70241fcb1b91 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:28.318799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-04466ec6-0144-4269-a17a-70241fcb1b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:28.327141 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c5037-73e1-4c41-b783-a91ded90d2b6 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:28.394656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:28.405876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c1ebf-9ce1-413b-8ef6-6b3636bc3721 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:28.407320 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c1ebf-9ce1-413b-8ef6-6b3636bc3721 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:28.407772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c1ebf-9ce1-413b-8ef6-6b3636bc3721 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:28.408188 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c1ebf-9ce1-413b-8ef6-6b3636bc3721 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:28.428778 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04466ec6-0144-4269-a17a-70241fcb1b91 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:28.434692 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70099bc-00f5-4803-91d8-00a595f44bc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:28.435636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70099bc-00f5-4803-91d8-00a595f44bc7 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:28.436354 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70099bc-00f5-4803-91d8-00a595f44bc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:28.437356 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70099bc-00f5-4803-91d8-00a595f44bc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:28.450565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70099bc-00f5-4803-91d8-00a595f44bc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:28.452394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70099bc-00f5-4803-91d8-00a595f44bc7 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:28.477075 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 952/1922] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:23:28.496144 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c1ebf-9ce1-413b-8ef6-6b3636bc3721 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:28.498131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c1ebf-9ce1-413b-8ef6-6b3636bc3721 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:28.514297 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 971/1923] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:23:28.626606 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70099bc-00f5-4803-91d8-00a595f44bc7 service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:28.639284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70099bc-00f5-4803-91d8-00a595f44bc7 service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:28.646804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294b5bd-cf32-4465-8ab8-210838400a58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:28.647433 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294b5bd-cf32-4465-8ab8-210838400a58 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:28.647712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294b5bd-cf32-4465-8ab8-210838400a58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:28.647984 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294b5bd-cf32-4465-8ab8-210838400a58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:28.651155 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c1ebf-9ce1-413b-8ef6-6b3636bc3721 service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:28.663416 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c1ebf-9ce1-413b-8ef6-6b3636bc3721 service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:28.667627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048b389-acd6-46c1-862a-8563387aa468 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:28.668352 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048b389-acd6-46c1-862a-8563387aa468 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:28.668633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048b389-acd6-46c1-862a-8563387aa468 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:28.668904 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048b389-acd6-46c1-862a-8563387aa468 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:28.731477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294b5bd-cf32-4465-8ab8-210838400a58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:28.733451 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294b5bd-cf32-4465-8ab8-210838400a58 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:28.746133 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 953/1924] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:23:28.757532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048b389-acd6-46c1-862a-8563387aa468 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:28.759137 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048b389-acd6-46c1-862a-8563387aa468 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:28.782523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 972/1925] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:23:38.717180 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b66ba2-9143-4235-baae-6a7ed2387275 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:38.718301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b66ba2-9143-4235-baae-6a7ed2387275 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:38.719068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b66ba2-9143-4235-baae-6a7ed2387275 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:38.720867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 954/1926] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 12:23:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:23:38.733073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd832202-e749-4065-b9d2-65586602b976 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:38.734039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd832202-e749-4065-b9d2-65586602b976 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:38.734459 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd832202-e749-4065-b9d2-65586602b976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:38.741041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=95,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:38.771007 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-bd832202-e749-4065-b9d2-65586602b976 None None] MFA Rules not processed for user `982b82e8ac424318a934f9149fb91706`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:38.772606 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-bd832202-e749-4065-b9d2-65586602b976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:38.954686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 973/1927] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 12:23:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:39.045491 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b66ba2-9143-4235-baae-6a7ed2387275 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:39.047609 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-95b66ba2-9143-4235-baae-6a7ed2387275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:39.090499 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b66ba2-9143-4235-baae-6a7ed2387275 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:39.102652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e9c43-63f3-48a0-a338-10dc20d93e32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:39.103394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e9c43-63f3-48a0-a338-10dc20d93e32 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:39.103739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e9c43-63f3-48a0-a338-10dc20d93e32 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:39.104074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e9c43-63f3-48a0-a338-10dc20d93e32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:39.115454 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e9c43-63f3-48a0-a338-10dc20d93e32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:39.117104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e9c43-63f3-48a0-a338-10dc20d93e32 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:39.131966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 955/1928] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:23:39.301825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd832202-e749-4065-b9d2-65586602b976 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:39.342305 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:39.386616 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd832202-e749-4065-b9d2-65586602b976 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:39.398503 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee29c80-23a5-4588-8843-1c9de59ac4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:39.399160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee29c80-23a5-4588-8843-1c9de59ac4df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:39.399439 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee29c80-23a5-4588-8843-1c9de59ac4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:39.399818 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee29c80-23a5-4588-8843-1c9de59ac4df admin admin] PATH_INFO: `/v3/users/95ad41337bad4a89bd77922bf4b52b93` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:39.400806 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee29c80-23a5-4588-8843-1c9de59ac4df admin admin] RBAC: Authorizing `identity:delete_user(user_id=95ad41337bad4a89bd77922bf4b52b93)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:39.402797 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee29c80-23a5-4588-8843-1c9de59ac4df admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:39.463158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-eee29c80-23a5-4588-8843-1c9de59ac4df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95ad41337bad4a89bd77922bf4b52b93'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:39.463903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-eee29c80-23a5-4588-8843-1c9de59ac4df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95ad41337bad4a89bd77922bf4b52b93'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:39.471844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-eee29c80-23a5-4588-8843-1c9de59ac4df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95ad41337bad4a89bd77922bf4b52b93'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:39.472939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-eee29c80-23a5-4588-8843-1c9de59ac4df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95ad41337bad4a89bd77922bf4b52b93'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:39.481405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-eee29c80-23a5-4588-8843-1c9de59ac4df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95ad41337bad4a89bd77922bf4b52b93'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:39.502034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 974/1929] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:23:39 2022] DELETE /identity/v3/users/95ad41337bad4a89bd77922bf4b52b93 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:39.506988 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a288016-a482-4a2b-820b-ff598d90bccd None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:39.507325 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a288016-a482-4a2b-820b-ff598d90bccd None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:39.507604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a288016-a482-4a2b-820b-ff598d90bccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:39.529388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-0a288016-a482-4a2b-820b-ff598d90bccd None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:39.530598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0a288016-a482-4a2b-820b-ff598d90bccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:39.596300 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 956/1930] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:39.606783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee29c80-23a5-4588-8843-1c9de59ac4df admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:39.608232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-eee29c80-23a5-4588-8843-1c9de59ac4df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:39.668143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee29c80-23a5-4588-8843-1c9de59ac4df admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:39.672110 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6803a5-7c8e-4274-bb17-0b4832469eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:39.672794 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6803a5-7c8e-4274-bb17-0b4832469eeb service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:39.673227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6803a5-7c8e-4274-bb17-0b4832469eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:39.673619 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6803a5-7c8e-4274-bb17-0b4832469eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:39.683779 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6803a5-7c8e-4274-bb17-0b4832469eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:39.685335 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6803a5-7c8e-4274-bb17-0b4832469eeb service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:39.695448 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 975/1931] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:23:39.997345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a288016-a482-4a2b-820b-ff598d90bccd None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:39.998823 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0a288016-a482-4a2b-820b-ff598d90bccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:40.015130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a288016-a482-4a2b-820b-ff598d90bccd None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:40.031404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f0bc1-ea94-4703-9f17-f1390f4a9f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:40.032142 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f0bc1-ea94-4703-9f17-f1390f4a9f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:40.032497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f0bc1-ea94-4703-9f17-f1390f4a9f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:40.032846 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f0bc1-ea94-4703-9f17-f1390f4a9f94 admin admin] PATH_INFO: `/v3/projects/ec1f3717b7b04b328256ef25c662eab0` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:40.034252 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f0bc1-ea94-4703-9f17-f1390f4a9f94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec1f3717b7b04b328256ef25c662eab0)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:40.039235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723f0bc1-ea94-4703-9f17-f1390f4a9f94 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:40.113477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-723f0bc1-ea94-4703-9f17-f1390f4a9f94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec1f3717b7b04b328256ef25c662eab0', 'request_id': 'req-723f0bc1-ea94-4703-9f17-f1390f4a9f94'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:40.124919 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-723f0bc1-ea94-4703-9f17-f1390f4a9f94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec1f3717b7b04b328256ef25c662eab0', 'request_id': 'req-723f0bc1-ea94-4703-9f17-f1390f4a9f94'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:40.147287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-723f0bc1-ea94-4703-9f17-f1390f4a9f94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec1f3717b7b04b328256ef25c662eab0', 'request_id': 'req-723f0bc1-ea94-4703-9f17-f1390f4a9f94'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:40.162425 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-723f0bc1-ea94-4703-9f17-f1390f4a9f94 admin admin] The token cache is being invalidate because project ec1f3717b7b04b328256ef25c662eab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:23:40.163380 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-723f0bc1-ea94-4703-9f17-f1390f4a9f94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:40.164155 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-723f0bc1-ea94-4703-9f17-f1390f4a9f94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:40.169269 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 957/1932] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:23:39 2022] DELETE /identity/v3/projects/ec1f3717b7b04b328256ef25c662eab0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:40.170560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6803a5-7c8e-4274-bb17-0b4832469eeb service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:40.183034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6803a5-7c8e-4274-bb17-0b4832469eeb service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:40.243261 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:40.272379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46542ad2-4f14-4251-a420-260dca54bce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:40.273650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46542ad2-4f14-4251-a420-260dca54bce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:40.273650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46542ad2-4f14-4251-a420-260dca54bce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:40.273914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46542ad2-4f14-4251-a420-260dca54bce6 admin admin] PATH_INFO: `/v3/users/7559d138e2f04fdb92e44e77cd4c9c4a` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:40.275178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46542ad2-4f14-4251-a420-260dca54bce6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7559d138e2f04fdb92e44e77cd4c9c4a)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:40.277073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46542ad2-4f14-4251-a420-260dca54bce6 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:40.337806 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-46542ad2-4f14-4251-a420-260dca54bce6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7559d138e2f04fdb92e44e77cd4c9c4a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:40.338414 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-46542ad2-4f14-4251-a420-260dca54bce6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7559d138e2f04fdb92e44e77cd4c9c4a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:40.345898 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-46542ad2-4f14-4251-a420-260dca54bce6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7559d138e2f04fdb92e44e77cd4c9c4a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:40.346712 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-46542ad2-4f14-4251-a420-260dca54bce6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7559d138e2f04fdb92e44e77cd4c9c4a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:40.355560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-46542ad2-4f14-4251-a420-260dca54bce6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7559d138e2f04fdb92e44e77cd4c9c4a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:40.374270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 976/1933] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:23:40 2022] DELETE /identity/v3/users/7559d138e2f04fdb92e44e77cd4c9c4a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:40.657729 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f0bc1-ea94-4703-9f17-f1390f4a9f94 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:40.659190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-723f0bc1-ea94-4703-9f17-f1390f4a9f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:40.672489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f0bc1-ea94-4703-9f17-f1390f4a9f94 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:40.732331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360bc21-17e0-4311-ac70-7ddbb6d79745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:40.732972 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360bc21-17e0-4311-ac70-7ddbb6d79745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:40.733288 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360bc21-17e0-4311-ac70-7ddbb6d79745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:40.733566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360bc21-17e0-4311-ac70-7ddbb6d79745 admin admin] PATH_INFO: `/v3/projects/4d8c0ae0fd7145cf9cd54b43423be2c7` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:40.734677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360bc21-17e0-4311-ac70-7ddbb6d79745 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d8c0ae0fd7145cf9cd54b43423be2c7)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:40.736573 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1360bc21-17e0-4311-ac70-7ddbb6d79745 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:40.748255 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:40.785063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1360bc21-17e0-4311-ac70-7ddbb6d79745 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d8c0ae0fd7145cf9cd54b43423be2c7', 'request_id': 'req-1360bc21-17e0-4311-ac70-7ddbb6d79745'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:40.791207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1360bc21-17e0-4311-ac70-7ddbb6d79745 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d8c0ae0fd7145cf9cd54b43423be2c7', 'request_id': 'req-1360bc21-17e0-4311-ac70-7ddbb6d79745'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:40.798637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1360bc21-17e0-4311-ac70-7ddbb6d79745 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d8c0ae0fd7145cf9cd54b43423be2c7', 'request_id': 'req-1360bc21-17e0-4311-ac70-7ddbb6d79745'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:40.809649 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1360bc21-17e0-4311-ac70-7ddbb6d79745 admin admin] The token cache is being invalidate because project 4d8c0ae0fd7145cf9cd54b43423be2c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:23:40.809934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1360bc21-17e0-4311-ac70-7ddbb6d79745 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:40.810555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1360bc21-17e0-4311-ac70-7ddbb6d79745 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:40.812062 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 958/1934] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:23:40 2022] DELETE /identity/v3/projects/4d8c0ae0fd7145cf9cd54b43423be2c7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:40.816862 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb46c4-abc5-47bb-88f7-7c66941abde1 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:40.817156 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb46c4-abc5-47bb-88f7-7c66941abde1 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:40.817441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb46c4-abc5-47bb-88f7-7c66941abde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:40.836532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-43bb46c4-abc5-47bb-88f7-7c66941abde1 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:40.837742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-43bb46c4-abc5-47bb-88f7-7c66941abde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:40.897772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 977/1935] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:40.905704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0425a44-9524-4c26-9cb6-8ff523b0a415 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:40.905985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0425a44-9524-4c26-9cb6-8ff523b0a415 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:40.906255 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0425a44-9524-4c26-9cb6-8ff523b0a415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:40.926843 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-b0425a44-9524-4c26-9cb6-8ff523b0a415 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:40.928021 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b0425a44-9524-4c26-9cb6-8ff523b0a415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:40.936043 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 959/1936] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:40.940305 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb46c4-abc5-47bb-88f7-7c66941abde1 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:40.951432 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb46c4-abc5-47bb-88f7-7c66941abde1 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:40.955995 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e76445-ded5-4744-bdf9-6272645409f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:40.956823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e76445-ded5-4744-bdf9-6272645409f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:40.957302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e76445-ded5-4744-bdf9-6272645409f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:40.957593 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e76445-ded5-4744-bdf9-6272645409f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:40.958234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e76445-ded5-4744-bdf9-6272645409f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:23:40.958536 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e76445-ded5-4744-bdf9-6272645409f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:40.960128 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e76445-ded5-4744-bdf9-6272645409f2 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:40.971812 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 978/1937] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:23:40.976858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0425a44-9524-4c26-9cb6-8ff523b0a415 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:40.986274 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0425a44-9524-4c26-9cb6-8ff523b0a415 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:40.994384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6de416-2f1b-4abf-a6ee-c2e424f97dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:40.995009 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6de416-2f1b-4abf-a6ee-c2e424f97dcf admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:40.995279 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6de416-2f1b-4abf-a6ee-c2e424f97dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:40.995583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6de416-2f1b-4abf-a6ee-c2e424f97dcf admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:40.996178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6de416-2f1b-4abf-a6ee-c2e424f97dcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:40.998090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6de416-2f1b-4abf-a6ee-c2e424f97dcf admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:41.030767 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 960/1938] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:23:40 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:23:41.031939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e76445-ded5-4744-bdf9-6272645409f2 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:41.043236 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e76445-ded5-4744-bdf9-6272645409f2 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:41.094600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:41.102848 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd425e-fa7b-485f-8174-cdf96271b27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:41.103466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd425e-fa7b-485f-8174-cdf96271b27e admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:41.103748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd425e-fa7b-485f-8174-cdf96271b27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:41.104022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd425e-fa7b-485f-8174-cdf96271b27e admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:41.104664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd425e-fa7b-485f-8174-cdf96271b27e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:41.106718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd425e-fa7b-485f-8174-cdf96271b27e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:41.132777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 979/1939] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:23:41 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:23:41.134046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6de416-2f1b-4abf-a6ee-c2e424f97dcf admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:41.147448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6de416-2f1b-4abf-a6ee-c2e424f97dcf admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:41.153061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b1ad1-458b-4323-a196-338d99d2d517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:41.153717 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b1ad1-458b-4323-a196-338d99d2d517 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:41.154004 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b1ad1-458b-4323-a196-338d99d2d517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:41.154284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b1ad1-458b-4323-a196-338d99d2d517 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:41.154973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b1ad1-458b-4323-a196-338d99d2d517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:41.156889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b1ad1-458b-4323-a196-338d99d2d517 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:41.168860 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 961/1940] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:23:41.172900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd425e-fa7b-485f-8174-cdf96271b27e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:41.188944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd425e-fa7b-485f-8174-cdf96271b27e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:41.194034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c4b42-c209-4fb3-8314-e4396b373329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:41.194852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c4b42-c209-4fb3-8314-e4396b373329 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:41.195193 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c4b42-c209-4fb3-8314-e4396b373329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:41.195528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c4b42-c209-4fb3-8314-e4396b373329 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:41.196229 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c4b42-c209-4fb3-8314-e4396b373329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:41.197952 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185c4b42-c209-4fb3-8314-e4396b373329 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:41.209156 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 980/1941] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:23:41.213318 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b1ad1-458b-4323-a196-338d99d2d517 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:41.224680 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b1ad1-458b-4323-a196-338d99d2d517 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:41.229307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7668c-1be4-487e-9d10-45b415bfd726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:41.229958 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7668c-1be4-487e-9d10-45b415bfd726 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:41.230239 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7668c-1be4-487e-9d10-45b415bfd726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:41.230512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7668c-1be4-487e-9d10-45b415bfd726 admin admin] PATH_INFO: `/v3/projects/0e96c15d58a047109cca4d693dfa85c9/users/d6a116e71bac4989961858e750b58060/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:41.244838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7668c-1be4-487e-9d10-45b415bfd726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e96c15d58a047109cca4d693dfa85c9, user_id=d6a116e71bac4989961858e750b58060, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:41.247507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7668c-1be4-487e-9d10-45b415bfd726 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:41.260564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 962/1942] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:41 2022] PUT /identity/v3/projects/0e96c15d58a047109cca4d693dfa85c9/users/d6a116e71bac4989961858e750b58060/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:41.271267 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14759d-ab13-482d-8c93-c943d7c29cda None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:41.271576 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14759d-ab13-482d-8c93-c943d7c29cda None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:41.271847 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14759d-ab13-482d-8c93-c943d7c29cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:41.302155 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-4e14759d-ab13-482d-8c93-c943d7c29cda None None] MFA Rules not processed for user `d6a116e71bac4989961858e750b58060`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:41.303509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-4e14759d-ab13-482d-8c93-c943d7c29cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:41.376068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 981/1943] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:41.381226 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7668c-1be4-487e-9d10-45b415bfd726 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:41.451664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7668c-1be4-487e-9d10-45b415bfd726 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:41.457851 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf6745-b64a-4b09-a4dd-b14ed534665c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:41.458512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf6745-b64a-4b09-a4dd-b14ed534665c admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:41.458864 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf6745-b64a-4b09-a4dd-b14ed534665c admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:41.459194 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf6745-b64a-4b09-a4dd-b14ed534665c admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:41.459891 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf6745-b64a-4b09-a4dd-b14ed534665c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:41.461873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf6745-b64a-4b09-a4dd-b14ed534665c admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:41.490308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 963/1944] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:23:41 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:23:41.497464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14759d-ab13-482d-8c93-c943d7c29cda None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:41.508979 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14759d-ab13-482d-8c93-c943d7c29cda None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:41.562943 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a35f3-3f27-4eb3-aee1-fc009363c120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:41.563545 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a35f3-3f27-4eb3-aee1-fc009363c120 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:41.563818 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a35f3-3f27-4eb3-aee1-fc009363c120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:41.564084 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a35f3-3f27-4eb3-aee1-fc009363c120 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:41.564701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a35f3-3f27-4eb3-aee1-fc009363c120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:41.567359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a35f3-3f27-4eb3-aee1-fc009363c120 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:41.587592 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 982/1945] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:23:41 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:23:41.591507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf6745-b64a-4b09-a4dd-b14ed534665c admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:41.601923 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf6745-b64a-4b09-a4dd-b14ed534665c admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:41.606398 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26840a21-d9d5-4d30-b6f9-221d8677a36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:41.606987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26840a21-d9d5-4d30-b6f9-221d8677a36c admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:41.607270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26840a21-d9d5-4d30-b6f9-221d8677a36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:41.607538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26840a21-d9d5-4d30-b6f9-221d8677a36c admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:41.608166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26840a21-d9d5-4d30-b6f9-221d8677a36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:41.609784 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26840a21-d9d5-4d30-b6f9-221d8677a36c admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:41.620056 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 964/1946] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:23:41.624016 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a35f3-3f27-4eb3-aee1-fc009363c120 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:41.635801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a35f3-3f27-4eb3-aee1-fc009363c120 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:41.641104 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da497594-1bae-4565-a04e-31dcb8da4828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:41.641843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da497594-1bae-4565-a04e-31dcb8da4828 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:41.642119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da497594-1bae-4565-a04e-31dcb8da4828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:41.642385 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da497594-1bae-4565-a04e-31dcb8da4828 admin admin] PATH_INFO: `/v3/projects/3784af1f68ac49e79586d845435de494/users/eb2e642541824f5cab3f7e6ace4d7d6b/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:41.660887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da497594-1bae-4565-a04e-31dcb8da4828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3784af1f68ac49e79586d845435de494, user_id=eb2e642541824f5cab3f7e6ace4d7d6b, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:41.664372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da497594-1bae-4565-a04e-31dcb8da4828 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:41.684087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 983/1947] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:41 2022] PUT /identity/v3/projects/3784af1f68ac49e79586d845435de494/users/eb2e642541824f5cab3f7e6ace4d7d6b/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:41.687810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26840a21-d9d5-4d30-b6f9-221d8677a36c admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:41.698429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26840a21-d9d5-4d30-b6f9-221d8677a36c admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:41.706113 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:41.756565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3d8ea-5053-4e3c-9526-587c9272a42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:41.757198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3d8ea-5053-4e3c-9526-587c9272a42a admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:41.757485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3d8ea-5053-4e3c-9526-587c9272a42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:41.757761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3d8ea-5053-4e3c-9526-587c9272a42a admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:41.758393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af3d8ea-5053-4e3c-9526-587c9272a42a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:41.759978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af3d8ea-5053-4e3c-9526-587c9272a42a admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:41.769735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 965/1948] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:23:41.774011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da497594-1bae-4565-a04e-31dcb8da4828 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:41.783971 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da497594-1bae-4565-a04e-31dcb8da4828 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:41.788774 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd3969-4668-40b9-9035-c9408bc78e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:41.789448 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd3969-4668-40b9-9035-c9408bc78e1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:41.789732 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd3969-4668-40b9-9035-c9408bc78e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:41.790006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd3969-4668-40b9-9035-c9408bc78e1a admin admin] PATH_INFO: `/v3/projects/3784af1f68ac49e79586d845435de494/users/eb2e642541824f5cab3f7e6ace4d7d6b/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:41.791887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd3969-4668-40b9-9035-c9408bc78e1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3784af1f68ac49e79586d845435de494, user_id=eb2e642541824f5cab3f7e6ace4d7d6b, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:41.794518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd3969-4668-40b9-9035-c9408bc78e1a admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:41.805409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 984/1949] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:41 2022] PUT /identity/v3/projects/3784af1f68ac49e79586d845435de494/users/eb2e642541824f5cab3f7e6ace4d7d6b/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:41.817385 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7060a-ed28-4c2b-a43f-3610812176db None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:41.817904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7060a-ed28-4c2b-a43f-3610812176db None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:41.818409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7060a-ed28-4c2b-a43f-3610812176db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:41.850409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-89b7060a-ed28-4c2b-a43f-3610812176db None None] MFA Rules not processed for user `eb2e642541824f5cab3f7e6ace4d7d6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:41.851629 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-89b7060a-ed28-4c2b-a43f-3610812176db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:41.920739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 966/1950] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:41.932248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd3969-4668-40b9-9035-c9408bc78e1a admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:41.933982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7bfd3969-4668-40b9-9035-c9408bc78e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:41.953351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:41.991517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd3969-4668-40b9-9035-c9408bc78e1a admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:41.995114 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582de6b-8878-4ea5-bb19-8f552d3b78a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:41.995712 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582de6b-8878-4ea5-bb19-8f552d3b78a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:41.995990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582de6b-8878-4ea5-bb19-8f552d3b78a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:41.996292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582de6b-8878-4ea5-bb19-8f552d3b78a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:42.006729 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582de6b-8878-4ea5-bb19-8f552d3b78a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:42.007711 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582de6b-8878-4ea5-bb19-8f552d3b78a6 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:42.017986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 985/1951] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:23:42.669927 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7060a-ed28-4c2b-a43f-3610812176db None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:42.745243 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7060a-ed28-4c2b-a43f-3610812176db None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:42.749935 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd90c1-5c07-4d5e-982c-297a042400c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:42.750576 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd90c1-5c07-4d5e-982c-297a042400c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:42.750858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd90c1-5c07-4d5e-982c-297a042400c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:42.751131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd90c1-5c07-4d5e-982c-297a042400c8 admin admin] PATH_INFO: `/v3/users/d6a116e71bac4989961858e750b58060` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:42.752251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd90c1-5c07-4d5e-982c-297a042400c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6a116e71bac4989961858e750b58060)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:42.755046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd90c1-5c07-4d5e-982c-297a042400c8 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:42.791382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:42.801984 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-54bd90c1-5c07-4d5e-982c-297a042400c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6a116e71bac4989961858e750b58060'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:42.802635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-54bd90c1-5c07-4d5e-982c-297a042400c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6a116e71bac4989961858e750b58060'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:42.814220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-54bd90c1-5c07-4d5e-982c-297a042400c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6a116e71bac4989961858e750b58060'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:42.821930 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-54bd90c1-5c07-4d5e-982c-297a042400c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6a116e71bac4989961858e750b58060'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:42.822547 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-54bd90c1-5c07-4d5e-982c-297a042400c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6a116e71bac4989961858e750b58060'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:42.834425 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 967/1952] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:23:42 2022] DELETE /identity/v3/users/d6a116e71bac4989961858e750b58060 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:42.839136 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06736153-06a2-4939-941c-c5ee48780709 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:42.839616 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06736153-06a2-4939-941c-c5ee48780709 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:42.839713 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06736153-06a2-4939-941c-c5ee48780709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:42.858337 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-06736153-06a2-4939-941c-c5ee48780709 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:42.859527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-06736153-06a2-4939-941c-c5ee48780709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:42.918815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 986/1953] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:42.929187 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd90c1-5c07-4d5e-982c-297a042400c8 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:42.930523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-54bd90c1-5c07-4d5e-982c-297a042400c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:42.986578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd90c1-5c07-4d5e-982c-297a042400c8 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:42.990882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ffc2f-25bf-4919-83aa-fb1f55785fb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:42.991494 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ffc2f-25bf-4919-83aa-fb1f55785fb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:42.991773 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ffc2f-25bf-4919-83aa-fb1f55785fb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:42.992048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ffc2f-25bf-4919-83aa-fb1f55785fb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:43.000870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ffc2f-25bf-4919-83aa-fb1f55785fb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:43.002322 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ffc2f-25bf-4919-83aa-fb1f55785fb5 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:43.011709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 968/1954] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:23:43.280876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06736153-06a2-4939-941c-c5ee48780709 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:43.282417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-06736153-06a2-4939-941c-c5ee48780709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:43.299980 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06736153-06a2-4939-941c-c5ee48780709 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:43.313325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31283d2-0e0f-4cb5-a189-594ffd635ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:43.313986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31283d2-0e0f-4cb5-a189-594ffd635ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:43.314269 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31283d2-0e0f-4cb5-a189-594ffd635ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:43.314544 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31283d2-0e0f-4cb5-a189-594ffd635ef8 admin admin] PATH_INFO: `/v3/projects/0e96c15d58a047109cca4d693dfa85c9` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:43.316025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31283d2-0e0f-4cb5-a189-594ffd635ef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e96c15d58a047109cca4d693dfa85c9)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:43.318280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31283d2-0e0f-4cb5-a189-594ffd635ef8 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:43.372989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d31283d2-0e0f-4cb5-a189-594ffd635ef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e96c15d58a047109cca4d693dfa85c9', 'request_id': 'req-d31283d2-0e0f-4cb5-a189-594ffd635ef8'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:43.379551 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d31283d2-0e0f-4cb5-a189-594ffd635ef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e96c15d58a047109cca4d693dfa85c9', 'request_id': 'req-d31283d2-0e0f-4cb5-a189-594ffd635ef8'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:43.387686 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d31283d2-0e0f-4cb5-a189-594ffd635ef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e96c15d58a047109cca4d693dfa85c9', 'request_id': 'req-d31283d2-0e0f-4cb5-a189-594ffd635ef8'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:43.398730 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d31283d2-0e0f-4cb5-a189-594ffd635ef8 admin admin] The token cache is being invalidate because project 0e96c15d58a047109cca4d693dfa85c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:23:43.399018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d31283d2-0e0f-4cb5-a189-594ffd635ef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:43.399642 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d31283d2-0e0f-4cb5-a189-594ffd635ef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:43.401172 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 987/1955] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:23:43 2022] DELETE /identity/v3/projects/0e96c15d58a047109cca4d693dfa85c9 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:43.405176 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ffc2f-25bf-4919-83aa-fb1f55785fb5 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:43.415351 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ffc2f-25bf-4919-83aa-fb1f55785fb5 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:43.474986 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c06af-825d-47c6-a068-7db7bd77e895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:43.475566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c06af-825d-47c6-a068-7db7bd77e895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:43.475844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c06af-825d-47c6-a068-7db7bd77e895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:43.476113 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c06af-825d-47c6-a068-7db7bd77e895 admin admin] PATH_INFO: `/v3/users/eb2e642541824f5cab3f7e6ace4d7d6b` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:43.477281 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c06af-825d-47c6-a068-7db7bd77e895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb2e642541824f5cab3f7e6ace4d7d6b)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:43.479221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c06af-825d-47c6-a068-7db7bd77e895 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:43.533021 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6d7c06af-825d-47c6-a068-7db7bd77e895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb2e642541824f5cab3f7e6ace4d7d6b'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:43.533713 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6d7c06af-825d-47c6-a068-7db7bd77e895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb2e642541824f5cab3f7e6ace4d7d6b'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:43.541333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:43.548342 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6d7c06af-825d-47c6-a068-7db7bd77e895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb2e642541824f5cab3f7e6ace4d7d6b'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:43.556114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6d7c06af-825d-47c6-a068-7db7bd77e895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb2e642541824f5cab3f7e6ace4d7d6b'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:43.556728 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6d7c06af-825d-47c6-a068-7db7bd77e895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb2e642541824f5cab3f7e6ace4d7d6b'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:43.569193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 969/1956] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:23:43 2022] DELETE /identity/v3/users/eb2e642541824f5cab3f7e6ace4d7d6b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:43.958382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31283d2-0e0f-4cb5-a189-594ffd635ef8 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:43.959844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d31283d2-0e0f-4cb5-a189-594ffd635ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:43.973867 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31283d2-0e0f-4cb5-a189-594ffd635ef8 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:44.015955 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:44.033629 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abfbbc-5299-4d85-9081-80dec54048dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:44.034293 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abfbbc-5299-4d85-9081-80dec54048dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:44.034572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abfbbc-5299-4d85-9081-80dec54048dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:44.034844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abfbbc-5299-4d85-9081-80dec54048dd admin admin] PATH_INFO: `/v3/projects/3784af1f68ac49e79586d845435de494` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:44.035950 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85abfbbc-5299-4d85-9081-80dec54048dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3784af1f68ac49e79586d845435de494)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:44.037880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85abfbbc-5299-4d85-9081-80dec54048dd admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:44.084727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-85abfbbc-5299-4d85-9081-80dec54048dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3784af1f68ac49e79586d845435de494', 'request_id': 'req-85abfbbc-5299-4d85-9081-80dec54048dd'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:44.091655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-85abfbbc-5299-4d85-9081-80dec54048dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3784af1f68ac49e79586d845435de494', 'request_id': 'req-85abfbbc-5299-4d85-9081-80dec54048dd'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:44.099435 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-85abfbbc-5299-4d85-9081-80dec54048dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3784af1f68ac49e79586d845435de494', 'request_id': 'req-85abfbbc-5299-4d85-9081-80dec54048dd'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:44.109583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-85abfbbc-5299-4d85-9081-80dec54048dd admin admin] The token cache is being invalidate because project 3784af1f68ac49e79586d845435de494 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:23:44.109873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-85abfbbc-5299-4d85-9081-80dec54048dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:44.110502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-85abfbbc-5299-4d85-9081-80dec54048dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:23:44.111968 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 988/1957] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:23:43 2022] DELETE /identity/v3/projects/3784af1f68ac49e79586d845435de494 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:23:44.117473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5505f-d10b-452d-82a7-25a238e1ddfb None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:44.117929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5505f-d10b-452d-82a7-25a238e1ddfb None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:44.118039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5505f-d10b-452d-82a7-25a238e1ddfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:44.138539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-34c5505f-d10b-452d-82a7-25a238e1ddfb None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:44.139726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-34c5505f-d10b-452d-82a7-25a238e1ddfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:44.199107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 970/1958] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:44.206818 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8032879-5ab0-41c0-a86b-b1616643d4b7 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:44.207104 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8032879-5ab0-41c0-a86b-b1616643d4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:44.207868 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8032879-5ab0-41c0-a86b-b1616643d4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:44.227844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-b8032879-5ab0-41c0-a86b-b1616643d4b7 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:44.229014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b8032879-5ab0-41c0-a86b-b1616643d4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:44.236716 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 989/1959] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:44.240725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5505f-d10b-452d-82a7-25a238e1ddfb None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:44.250156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5505f-d10b-452d-82a7-25a238e1ddfb None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:44.254633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf273890-5822-4313-a594-5a95eebfb231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:44.255221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf273890-5822-4313-a594-5a95eebfb231 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:44.255498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf273890-5822-4313-a594-5a95eebfb231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:44.255766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf273890-5822-4313-a594-5a95eebfb231 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:44.256400 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf273890-5822-4313-a594-5a95eebfb231 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:23:44.256699 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf273890-5822-4313-a594-5a95eebfb231 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:44.258298 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf273890-5822-4313-a594-5a95eebfb231 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:44.269548 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 971/1960] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:23:44.274464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8032879-5ab0-41c0-a86b-b1616643d4b7 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:44.283577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8032879-5ab0-41c0-a86b-b1616643d4b7 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:44.289284 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e15d38-be28-4103-a88e-61d6a6324698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:44.289902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e15d38-be28-4103-a88e-61d6a6324698 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:44.290184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e15d38-be28-4103-a88e-61d6a6324698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:44.290460 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e15d38-be28-4103-a88e-61d6a6324698 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:44.291094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e15d38-be28-4103-a88e-61d6a6324698 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:44.292970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e15d38-be28-4103-a88e-61d6a6324698 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:44.320241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 990/1961] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:23:44 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:23:44.324360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf273890-5822-4313-a594-5a95eebfb231 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:44.334978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf273890-5822-4313-a594-5a95eebfb231 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:44.400471 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5198bc-006a-4e25-affb-dac43a1c0a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:44.401200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5198bc-006a-4e25-affb-dac43a1c0a1b admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:44.401489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5198bc-006a-4e25-affb-dac43a1c0a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:44.401769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5198bc-006a-4e25-affb-dac43a1c0a1b admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:44.402414 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5198bc-006a-4e25-affb-dac43a1c0a1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:44.404329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5198bc-006a-4e25-affb-dac43a1c0a1b admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:44.427799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 972/1962] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:23:44 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:23:44.432124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e15d38-be28-4103-a88e-61d6a6324698 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:44.442855 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e15d38-be28-4103-a88e-61d6a6324698 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:44.447687 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d7cc8-2cb8-413e-93b7-de73687b0332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:44.448292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d7cc8-2cb8-413e-93b7-de73687b0332 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:44.448569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d7cc8-2cb8-413e-93b7-de73687b0332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:44.448835 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d7cc8-2cb8-413e-93b7-de73687b0332 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:44.449518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d7cc8-2cb8-413e-93b7-de73687b0332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:44.451177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d7cc8-2cb8-413e-93b7-de73687b0332 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:44.462224 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 991/1963] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:23:44.466871 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5198bc-006a-4e25-affb-dac43a1c0a1b admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:44.477907 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5198bc-006a-4e25-affb-dac43a1c0a1b admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:44.482601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec57e0-c546-4083-93fa-9a61d4ca5339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:44.483328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec57e0-c546-4083-93fa-9a61d4ca5339 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:44.483653 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec57e0-c546-4083-93fa-9a61d4ca5339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:44.483966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec57e0-c546-4083-93fa-9a61d4ca5339 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:44.484720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec57e0-c546-4083-93fa-9a61d4ca5339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:44.486423 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec57e0-c546-4083-93fa-9a61d4ca5339 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:44.498169 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 973/1964] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:23:44.502261 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d7cc8-2cb8-413e-93b7-de73687b0332 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:44.513499 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d7cc8-2cb8-413e-93b7-de73687b0332 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:44.518353 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5592f97-2511-42b7-ab38-af1268a0b917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:44.519018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5592f97-2511-42b7-ab38-af1268a0b917 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:44.519296 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5592f97-2511-42b7-ab38-af1268a0b917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:44.519568 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5592f97-2511-42b7-ab38-af1268a0b917 admin admin] PATH_INFO: `/v3/projects/bb95e933d04c4042a3db6af294db4f10/users/64000daed6ad4cc8b4d4eb12cb080c49/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:44.537318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5592f97-2511-42b7-ab38-af1268a0b917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb95e933d04c4042a3db6af294db4f10, user_id=64000daed6ad4cc8b4d4eb12cb080c49, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:44.540233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5592f97-2511-42b7-ab38-af1268a0b917 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:44.559619 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 992/1965] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:23:44 2022] PUT /identity/v3/projects/bb95e933d04c4042a3db6af294db4f10/users/64000daed6ad4cc8b4d4eb12cb080c49/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:23:44.573067 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25998e53-ec7e-4aa0-8af4-69efdc7f2a8f None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:44.573505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25998e53-ec7e-4aa0-8af4-69efdc7f2a8f None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:44.573772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25998e53-ec7e-4aa0-8af4-69efdc7f2a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:44.612057 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-25998e53-ec7e-4aa0-8af4-69efdc7f2a8f None None] MFA Rules not processed for user `64000daed6ad4cc8b4d4eb12cb080c49`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:23:44.613830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-25998e53-ec7e-4aa0-8af4-69efdc7f2a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:44.654994 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:44.705826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 974/1966] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:23:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:23:44.719837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5592f97-2511-42b7-ab38-af1268a0b917 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:23:44.721915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-c5592f97-2511-42b7-ab38-af1268a0b917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:23:44.798419 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5592f97-2511-42b7-ab38-af1268a0b917 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:23:44.803276 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5331ce-28e2-495f-b3d4-4fe4486b4c7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:23:44.803972 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5331ce-28e2-495f-b3d4-4fe4486b4c7f service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:23:44.804359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5331ce-28e2-495f-b3d4-4fe4486b4c7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:23:44.804615 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5331ce-28e2-495f-b3d4-4fe4486b4c7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:23:44.816464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5331ce-28e2-495f-b3d4-4fe4486b4c7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:23:44.819553 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5331ce-28e2-495f-b3d4-4fe4486b4c7f service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:23:44.834303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 993/1967] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:23:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:23:54.702204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=14 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:23:54.828461 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:00.037732 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25998e53-ec7e-4aa0-8af4-69efdc7f2a8f None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:00.039192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-25998e53-ec7e-4aa0-8af4-69efdc7f2a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:00.110401 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25998e53-ec7e-4aa0-8af4-69efdc7f2a8f None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:00.115796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5331ce-28e2-495f-b3d4-4fe4486b4c7f service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:00.117833 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518dbe8-bfc0-4ddd-ae06-a2e999f8096c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:00.118480 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518dbe8-bfc0-4ddd-ae06-a2e999f8096c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:00.118759 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518dbe8-bfc0-4ddd-ae06-a2e999f8096c admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:00.119034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518dbe8-bfc0-4ddd-ae06-a2e999f8096c admin admin] PATH_INFO: `/v3/users/6a66eb3589d2422ab7ec898f35f54242` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:00.120193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518dbe8-bfc0-4ddd-ae06-a2e999f8096c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a66eb3589d2422ab7ec898f35f54242)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:00.122334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518dbe8-bfc0-4ddd-ae06-a2e999f8096c admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:00.126612 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5331ce-28e2-495f-b3d4-4fe4486b4c7f service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:00.131928 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14290413-e505-47ce-a4b5-a439ec043d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:00.132559 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14290413-e505-47ce-a4b5-a439ec043d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:00.132840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14290413-e505-47ce-a4b5-a439ec043d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:00.133193 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14290413-e505-47ce-a4b5-a439ec043d78 admin admin] PATH_INFO: `/v3/users/64000daed6ad4cc8b4d4eb12cb080c49` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:00.134301 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14290413-e505-47ce-a4b5-a439ec043d78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64000daed6ad4cc8b4d4eb12cb080c49)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:00.136244 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14290413-e505-47ce-a4b5-a439ec043d78 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:00.172752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-7518dbe8-bfc0-4ddd-ae06-a2e999f8096c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a66eb3589d2422ab7ec898f35f54242'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.173463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-7518dbe8-bfc0-4ddd-ae06-a2e999f8096c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a66eb3589d2422ab7ec898f35f54242'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.189890 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-14290413-e505-47ce-a4b5-a439ec043d78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64000daed6ad4cc8b4d4eb12cb080c49'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.190719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-14290413-e505-47ce-a4b5-a439ec043d78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64000daed6ad4cc8b4d4eb12cb080c49'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.195205 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-7518dbe8-bfc0-4ddd-ae06-a2e999f8096c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a66eb3589d2422ab7ec898f35f54242'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.200157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-14290413-e505-47ce-a4b5-a439ec043d78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64000daed6ad4cc8b4d4eb12cb080c49'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.200973 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-14290413-e505-47ce-a4b5-a439ec043d78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64000daed6ad4cc8b4d4eb12cb080c49'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.203633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-7518dbe8-bfc0-4ddd-ae06-a2e999f8096c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a66eb3589d2422ab7ec898f35f54242'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.204316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-7518dbe8-bfc0-4ddd-ae06-a2e999f8096c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a66eb3589d2422ab7ec898f35f54242'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.209865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-14290413-e505-47ce-a4b5-a439ec043d78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64000daed6ad4cc8b4d4eb12cb080c49'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.221915 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 975/1968] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:00 2022] DELETE /identity/v3/users/6a66eb3589d2422ab7ec898f35f54242 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:00.227199 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dbbfb-a9ad-4648-abb3-96abbb663f3b None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:00.227529 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dbbfb-a9ad-4648-abb3-96abbb663f3b None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:00.227842 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704dbbfb-a9ad-4648-abb3-96abbb663f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:00.232224 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 994/1969] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:00 2022] DELETE /identity/v3/users/64000daed6ad4cc8b4d4eb12cb080c49 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:24:00.236295 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92213fa5-bb67-4a77-a0e7-2dc7fa8e8601 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:00.236585 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92213fa5-bb67-4a77-a0e7-2dc7fa8e8601 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:00.236906 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92213fa5-bb67-4a77-a0e7-2dc7fa8e8601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:00.257717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-92213fa5-bb67-4a77-a0e7-2dc7fa8e8601 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:00.258911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-92213fa5-bb67-4a77-a0e7-2dc7fa8e8601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:00.264706 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-704dbbfb-a9ad-4648-abb3-96abbb663f3b None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:00.266237 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-704dbbfb-a9ad-4648-abb3-96abbb663f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:00.327137 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 995/1970] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:00.333591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92213fa5-bb67-4a77-a0e7-2dc7fa8e8601 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:00.335096 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-92213fa5-bb67-4a77-a0e7-2dc7fa8e8601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:00.337979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 976/1971] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:00.349158 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dbbfb-a9ad-4648-abb3-96abbb663f3b None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:00.350822 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-704dbbfb-a9ad-4648-abb3-96abbb663f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:00.395175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:00.416984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92213fa5-bb67-4a77-a0e7-2dc7fa8e8601 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:00.421499 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de1983-095a-42c6-bb14-e37133b09120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:00.422413 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de1983-095a-42c6-bb14-e37133b09120 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:00.422714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de1983-095a-42c6-bb14-e37133b09120 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:00.423042 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de1983-095a-42c6-bb14-e37133b09120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:00.435148 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de1983-095a-42c6-bb14-e37133b09120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:00.437386 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de1983-095a-42c6-bb14-e37133b09120 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:00.454486 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704dbbfb-a9ad-4648-abb3-96abbb663f3b None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:00.455215 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 996/1972] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:00.461507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c656ff2-3965-441d-bc0b-c216b21bdc23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:00.462372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c656ff2-3965-441d-bc0b-c216b21bdc23 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:00.462654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c656ff2-3965-441d-bc0b-c216b21bdc23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:00.463260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c656ff2-3965-441d-bc0b-c216b21bdc23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:00.474129 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c656ff2-3965-441d-bc0b-c216b21bdc23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:00.476004 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c656ff2-3965-441d-bc0b-c216b21bdc23 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:00.488059 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 977/1973] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:00.852820 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de1983-095a-42c6-bb14-e37133b09120 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:00.854428 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-53de1983-095a-42c6-bb14-e37133b09120 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:00.884774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c656ff2-3965-441d-bc0b-c216b21bdc23 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:00.886811 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2c656ff2-3965-441d-bc0b-c216b21bdc23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:00.889198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de1983-095a-42c6-bb14-e37133b09120 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:00.898123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98f61d-ad61-4ef7-99bb-f47c24483584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:00.898791 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98f61d-ad61-4ef7-99bb-f47c24483584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:00.899039 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98f61d-ad61-4ef7-99bb-f47c24483584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:00.899308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98f61d-ad61-4ef7-99bb-f47c24483584 admin admin] PATH_INFO: `/v3/projects/bb95e933d04c4042a3db6af294db4f10` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:00.901187 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf98f61d-ad61-4ef7-99bb-f47c24483584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb95e933d04c4042a3db6af294db4f10)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:00.902112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c656ff2-3965-441d-bc0b-c216b21bdc23 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:00.903446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf98f61d-ad61-4ef7-99bb-f47c24483584 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:00.907554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4af06a-e4ad-4f08-a4f3-846d952c0ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:00.908167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4af06a-e4ad-4f08-a4f3-846d952c0ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:00.908527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4af06a-e4ad-4f08-a4f3-846d952c0ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:00.908895 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4af06a-e4ad-4f08-a4f3-846d952c0ab1 admin admin] PATH_INFO: `/v3/projects/6a5863f760bf42a2971567455ecf4a61` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:00.910400 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4af06a-e4ad-4f08-a4f3-846d952c0ab1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a5863f760bf42a2971567455ecf4a61)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:00.912720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4af06a-e4ad-4f08-a4f3-846d952c0ab1 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:00.916322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:00.957691 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-bf98f61d-ad61-4ef7-99bb-f47c24483584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb95e933d04c4042a3db6af294db4f10', 'request_id': 'req-bf98f61d-ad61-4ef7-99bb-f47c24483584'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.965240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-bf98f61d-ad61-4ef7-99bb-f47c24483584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb95e933d04c4042a3db6af294db4f10', 'request_id': 'req-bf98f61d-ad61-4ef7-99bb-f47c24483584'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.966146 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ce4af06a-e4ad-4f08-a4f3-846d952c0ab1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a5863f760bf42a2971567455ecf4a61', 'request_id': 'req-ce4af06a-e4ad-4f08-a4f3-846d952c0ab1'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.972799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ce4af06a-e4ad-4f08-a4f3-846d952c0ab1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a5863f760bf42a2971567455ecf4a61', 'request_id': 'req-ce4af06a-e4ad-4f08-a4f3-846d952c0ab1'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.973885 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-bf98f61d-ad61-4ef7-99bb-f47c24483584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb95e933d04c4042a3db6af294db4f10', 'request_id': 'req-bf98f61d-ad61-4ef7-99bb-f47c24483584'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.982064 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ce4af06a-e4ad-4f08-a4f3-846d952c0ab1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a5863f760bf42a2971567455ecf4a61', 'request_id': 'req-ce4af06a-e4ad-4f08-a4f3-846d952c0ab1'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.986254 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-bf98f61d-ad61-4ef7-99bb-f47c24483584 admin admin] The token cache is being invalidate because project bb95e933d04c4042a3db6af294db4f10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:00.987128 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-bf98f61d-ad61-4ef7-99bb-f47c24483584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.987741 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-bf98f61d-ad61-4ef7-99bb-f47c24483584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.989509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 997/1974] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:00 2022] DELETE /identity/v3/projects/bb95e933d04c4042a3db6af294db4f10 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:00.994507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ce4af06a-e4ad-4f08-a4f3-846d952c0ab1 admin admin] The token cache is being invalidate because project 6a5863f760bf42a2971567455ecf4a61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:00.994669 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279781a-8880-493f-a798-4d97e562dbe4 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:00.994798 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ce4af06a-e4ad-4f08-a4f3-846d952c0ab1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.994919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279781a-8880-493f-a798-4d97e562dbe4 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:00.995207 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279781a-8880-493f-a798-4d97e562dbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:00.995466 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ce4af06a-e4ad-4f08-a4f3-846d952c0ab1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:00.997180 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 978/1975] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:00 2022] DELETE /identity/v3/projects/6a5863f760bf42a2971567455ecf4a61 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:01.001605 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ac422-8a0a-44ce-a901-807721ade366 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.001891 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ac422-8a0a-44ce-a901-807721ade366 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.002165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ac422-8a0a-44ce-a901-807721ade366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.018017 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7279781a-8880-493f-a798-4d97e562dbe4 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:01.019353 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7279781a-8880-493f-a798-4d97e562dbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:01.028604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-b86ac422-8a0a-44ce-a901-807721ade366 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:01.029886 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b86ac422-8a0a-44ce-a901-807721ade366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:01.099582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 979/1976] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:01.102301 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 998/1977] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:01.109661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75929cfa-1fc8-427f-b5d8-2fab09c1a7b4 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.109953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75929cfa-1fc8-427f-b5d8-2fab09c1a7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.110233 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75929cfa-1fc8-427f-b5d8-2fab09c1a7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.110391 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d77437-0e15-4a71-b651-3e46d5417621 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.110762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d77437-0e15-4a71-b651-3e46d5417621 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.111103 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d77437-0e15-4a71-b651-3e46d5417621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.131287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-75929cfa-1fc8-427f-b5d8-2fab09c1a7b4 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:01.131933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-86d77437-0e15-4a71-b651-3e46d5417621 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:01.132722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-75929cfa-1fc8-427f-b5d8-2fab09c1a7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:01.133253 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-86d77437-0e15-4a71-b651-3e46d5417621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:01.140890 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 980/1978] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:01.144423 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 999/1979] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:01.146084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75929cfa-1fc8-427f-b5d8-2fab09c1a7b4 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:01.146390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d77437-0e15-4a71-b651-3e46d5417621 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:01.156074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d77437-0e15-4a71-b651-3e46d5417621 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:01.156816 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75929cfa-1fc8-427f-b5d8-2fab09c1a7b4 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:01.161491 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf17ae3-52fc-4e1b-b8d9-88f1542d914f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:01.162220 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf17ae3-52fc-4e1b-b8d9-88f1542d914f admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.162572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf17ae3-52fc-4e1b-b8d9-88f1542d914f admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.162848 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf17ae3-52fc-4e1b-b8d9-88f1542d914f admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.163387 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8121f3e7-75f0-427b-8e66-4fd3c7f84e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:01.163543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf17ae3-52fc-4e1b-b8d9-88f1542d914f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:24:01.163846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf17ae3-52fc-4e1b-b8d9-88f1542d914f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:01.164087 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121f3e7-75f0-427b-8e66-4fd3c7f84e0a admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.164352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121f3e7-75f0-427b-8e66-4fd3c7f84e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.164731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121f3e7-75f0-427b-8e66-4fd3c7f84e0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.165423 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8121f3e7-75f0-427b-8e66-4fd3c7f84e0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:24:01.165908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8121f3e7-75f0-427b-8e66-4fd3c7f84e0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:01.166031 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf17ae3-52fc-4e1b-b8d9-88f1542d914f admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:01.167748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8121f3e7-75f0-427b-8e66-4fd3c7f84e0a admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:01.183976 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1000/1980] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:24:01.184153 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 981/1981] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:24:01.185124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf17ae3-52fc-4e1b-b8d9-88f1542d914f admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:01.188079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8121f3e7-75f0-427b-8e66-4fd3c7f84e0a admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:01.195727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf17ae3-52fc-4e1b-b8d9-88f1542d914f admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:01.199344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8121f3e7-75f0-427b-8e66-4fd3c7f84e0a admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:01.200702 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44283b-3d99-4f1e-b6b3-2a5cd2f3ebd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:01.201344 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44283b-3d99-4f1e-b6b3-2a5cd2f3ebd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.201653 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44283b-3d99-4f1e-b6b3-2a5cd2f3ebd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.201930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44283b-3d99-4f1e-b6b3-2a5cd2f3ebd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.202557 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44283b-3d99-4f1e-b6b3-2a5cd2f3ebd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:01.204278 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b484771-1ff3-46af-b47a-2a5b4280018a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:01.204431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44283b-3d99-4f1e-b6b3-2a5cd2f3ebd9 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:01.204876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b484771-1ff3-46af-b47a-2a5b4280018a admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.205215 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b484771-1ff3-46af-b47a-2a5b4280018a admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.205460 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b484771-1ff3-46af-b47a-2a5b4280018a admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.206099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b484771-1ff3-46af-b47a-2a5b4280018a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:01.208044 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b484771-1ff3-46af-b47a-2a5b4280018a admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:01.232217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1001/1982] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:01 2022] POST /identity/v3/projects => generated 397 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:01.236461 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44283b-3d99-4f1e-b6b3-2a5cd2f3ebd9 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:01.237512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 982/1983] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:01 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:01.241984 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b484771-1ff3-46af-b47a-2a5b4280018a admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:01.248232 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:01.300416 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:01.304745 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44283b-3d99-4f1e-b6b3-2a5cd2f3ebd9 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:01.322144 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b484771-1ff3-46af-b47a-2a5b4280018a admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:01.326940 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c79a3-86df-46ed-9878-85406d4baba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:01.327555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c79a3-86df-46ed-9878-85406d4baba5 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.327842 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c79a3-86df-46ed-9878-85406d4baba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.328120 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c79a3-86df-46ed-9878-85406d4baba5 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.328749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c79a3-86df-46ed-9878-85406d4baba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:01.330713 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c79a3-86df-46ed-9878-85406d4baba5 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:01.352829 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 983/1984] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:01 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:01.353835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c79a3-86df-46ed-9878-85406d4baba5 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:01.355434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-389c79a3-86df-46ed-9878-85406d4baba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:01.368190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c79a3-86df-46ed-9878-85406d4baba5 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:01.368349 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5b451-40d8-42ff-879a-f1897c7f5c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:01.369029 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5b451-40d8-42ff-879a-f1897c7f5c91 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.369362 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5b451-40d8-42ff-879a-f1897c7f5c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.369638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5b451-40d8-42ff-879a-f1897c7f5c91 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.370267 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5b451-40d8-42ff-879a-f1897c7f5c91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:01.372151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5b451-40d8-42ff-879a-f1897c7f5c91 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:01.372840 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0226f2-bccd-4ee4-a12a-056fe043aeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:01.373494 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0226f2-bccd-4ee4-a12a-056fe043aeb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.373772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0226f2-bccd-4ee4-a12a-056fe043aeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.374042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0226f2-bccd-4ee4-a12a-056fe043aeb6 admin admin] PATH_INFO: `/v3/users/2ce30508a72f478b933e4bcc0a4153a9` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.375162 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0226f2-bccd-4ee4-a12a-056fe043aeb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ce30508a72f478b933e4bcc0a4153a9)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:01.377033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0226f2-bccd-4ee4-a12a-056fe043aeb6 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:01.395189 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1002/1985] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:01 2022] POST /identity/v3/users => generated 299 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:01.396442 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5b451-40d8-42ff-879a-f1897c7f5c91 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:01.406231 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5b451-40d8-42ff-879a-f1897c7f5c91 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:01.411690 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6958db-de9c-4072-a4e6-7f2f10f0ef9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:01.412451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6958db-de9c-4072-a4e6-7f2f10f0ef9e admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.412758 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6958db-de9c-4072-a4e6-7f2f10f0ef9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.413119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6958db-de9c-4072-a4e6-7f2f10f0ef9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.413882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6958db-de9c-4072-a4e6-7f2f10f0ef9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:01.415836 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6958db-de9c-4072-a4e6-7f2f10f0ef9e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:01.427333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1003/1986] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:01.428525 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6958db-de9c-4072-a4e6-7f2f10f0ef9e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:01.431099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-0a0226f2-bccd-4ee4-a12a-056fe043aeb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ce30508a72f478b933e4bcc0a4153a9'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:01.432104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-0a0226f2-bccd-4ee4-a12a-056fe043aeb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ce30508a72f478b933e4bcc0a4153a9'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:01.440100 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6958db-de9c-4072-a4e6-7f2f10f0ef9e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:01.445956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-0a0226f2-bccd-4ee4-a12a-056fe043aeb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ce30508a72f478b933e4bcc0a4153a9'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:01.446132 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d4d3e-f30c-4f6b-864c-0861716067f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:01.446731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d4d3e-f30c-4f6b-864c-0861716067f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.447007 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d4d3e-f30c-4f6b-864c-0861716067f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.447410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d4d3e-f30c-4f6b-864c-0861716067f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.448097 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d4d3e-f30c-4f6b-864c-0861716067f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:01.449775 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d4d3e-f30c-4f6b-864c-0861716067f0 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:01.455033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-0a0226f2-bccd-4ee4-a12a-056fe043aeb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ce30508a72f478b933e4bcc0a4153a9'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:01.456006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-0a0226f2-bccd-4ee4-a12a-056fe043aeb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ce30508a72f478b933e4bcc0a4153a9'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:01.464827 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1004/1987] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:01.466365 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d4d3e-f30c-4f6b-864c-0861716067f0 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:01.468085 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-340d4d3e-f30c-4f6b-864c-0861716067f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:01.475615 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 984/1988] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:01 2022] DELETE /identity/v3/users/2ce30508a72f478b933e4bcc0a4153a9 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:01.476419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0226f2-bccd-4ee4-a12a-056fe043aeb6 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:01.478013 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0a0226f2-bccd-4ee4-a12a-056fe043aeb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:01.483616 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d4d3e-f30c-4f6b-864c-0861716067f0 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:01.553895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabb41e-d402-4430-a30a-f6b846aa76c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:01.554563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabb41e-d402-4430-a30a-f6b846aa76c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.554945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabb41e-d402-4430-a30a-f6b846aa76c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.555281 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabb41e-d402-4430-a30a-f6b846aa76c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.556412 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabb41e-d402-4430-a30a-f6b846aa76c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:01.558751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabb41e-d402-4430-a30a-f6b846aa76c1 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:01.571724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1005/1989] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:01.574985 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a8f8a-64f9-4fec-9075-e2d6870a26dc None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.574985 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a8f8a-64f9-4fec-9075-e2d6870a26dc None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.574985 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a8f8a-64f9-4fec-9075-e2d6870a26dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.580442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0226f2-bccd-4ee4-a12a-056fe043aeb6 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:01.586239 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa25201-2abf-4b13-a4f2-f33fcd74b930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:01.586580 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa25201-2abf-4b13-a4f2-f33fcd74b930 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.586864 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa25201-2abf-4b13-a4f2-f33fcd74b930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.587281 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa25201-2abf-4b13-a4f2-f33fcd74b930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.587928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa25201-2abf-4b13-a4f2-f33fcd74b930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:01.589563 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa25201-2abf-4b13-a4f2-f33fcd74b930 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:01.597325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-315a8f8a-64f9-4fec-9075-e2d6870a26dc None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:01.599442 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-315a8f8a-64f9-4fec-9075-e2d6870a26dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:01.601096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 985/1990] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:01.602333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa25201-2abf-4b13-a4f2-f33fcd74b930 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:01.608028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1006/1991] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:01.608163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:01.608818 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a8f8a-64f9-4fec-9075-e2d6870a26dc None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:01.616667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa25201-2abf-4b13-a4f2-f33fcd74b930 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:01.624502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a8f8a-64f9-4fec-9075-e2d6870a26dc None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:01.625332 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3363a149-f1de-4934-bb82-e4fc242c2eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:01.626168 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363a149-f1de-4934-bb82-e4fc242c2eae admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.626643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363a149-f1de-4934-bb82-e4fc242c2eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.627076 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3363a149-f1de-4934-bb82-e4fc242c2eae admin admin] PATH_INFO: `/v3/projects/12f9a758d17647ebaec796b73ba831f8/users/badc483a47a8455fb91dabbec8b64831/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.629859 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c6b74-2e3f-4564-ac81-b442e993bb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:01.630525 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c6b74-2e3f-4564-ac81-b442e993bb1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.630804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c6b74-2e3f-4564-ac81-b442e993bb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.631072 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c6b74-2e3f-4564-ac81-b442e993bb1d admin admin] PATH_INFO: `/v3/projects/a3e85056b9b84206b02ab50803269765/users/dc63407e186745fea7225f08f9f693c4/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.646204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3363a149-f1de-4934-bb82-e4fc242c2eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=12f9a758d17647ebaec796b73ba831f8, user_id=badc483a47a8455fb91dabbec8b64831, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:01.646844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c6b74-2e3f-4564-ac81-b442e993bb1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e85056b9b84206b02ab50803269765, user_id=dc63407e186745fea7225f08f9f693c4, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:01.649619 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c6b74-2e3f-4564-ac81-b442e993bb1d admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:01.656489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3363a149-f1de-4934-bb82-e4fc242c2eae admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:01.666934 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1007/1992] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:01 2022] PUT /identity/v3/projects/a3e85056b9b84206b02ab50803269765/users/dc63407e186745fea7225f08f9f693c4/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:01.667909 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c6b74-2e3f-4564-ac81-b442e993bb1d admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:01.673788 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 986/1993] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:01 2022] PUT /identity/v3/projects/12f9a758d17647ebaec796b73ba831f8/users/badc483a47a8455fb91dabbec8b64831/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:01.684035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d4c4a-8f39-4cf1-afcd-ebdc90d3e585 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.684409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d4c4a-8f39-4cf1-afcd-ebdc90d3e585 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.684729 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d4c4a-8f39-4cf1-afcd-ebdc90d3e585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.690087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-266c6b74-2e3f-4564-ac81-b442e993bb1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:01.722065 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-114d4c4a-8f39-4cf1-afcd-ebdc90d3e585 None None] MFA Rules not processed for user `badc483a47a8455fb91dabbec8b64831`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:01.723928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-114d4c4a-8f39-4cf1-afcd-ebdc90d3e585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:01.736548 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:01.761198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c6b74-2e3f-4564-ac81-b442e993bb1d admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:01.821456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 987/1994] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:01.822732 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d06f5-58ec-4eff-b9c2-b0d51c4b0251 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.823016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d06f5-58ec-4eff-b9c2-b0d51c4b0251 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.823726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d06f5-58ec-4eff-b9c2-b0d51c4b0251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.827543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287f998-93e3-4507-8f56-17e3429ba384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:01.829323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287f998-93e3-4507-8f56-17e3429ba384 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:01.829842 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287f998-93e3-4507-8f56-17e3429ba384 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:01.830287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287f998-93e3-4507-8f56-17e3429ba384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:01.870272 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-074d06f5-58ec-4eff-b9c2-b0d51c4b0251 None None] MFA Rules not processed for user `dc63407e186745fea7225f08f9f693c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:01.871910 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-074d06f5-58ec-4eff-b9c2-b0d51c4b0251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:01.915118 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287f998-93e3-4507-8f56-17e3429ba384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:01.917307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9287f998-93e3-4507-8f56-17e3429ba384 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:01.942338 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1008/1995] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:01.942526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287f998-93e3-4507-8f56-17e3429ba384 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:01.947499 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9287f998-93e3-4507-8f56-17e3429ba384 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:01.962012 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 988/1996] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:01 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:02.031463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d06f5-58ec-4eff-b9c2-b0d51c4b0251 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:02.033122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-074d06f5-58ec-4eff-b9c2-b0d51c4b0251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:02.098834 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287f998-93e3-4507-8f56-17e3429ba384 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:02.102798 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21f014-423c-47f1-840a-504eb9fa67a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:02.104022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21f014-423c-47f1-840a-504eb9fa67a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:02.104313 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21f014-423c-47f1-840a-504eb9fa67a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:02.104651 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21f014-423c-47f1-840a-504eb9fa67a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:02.115371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b21f014-423c-47f1-840a-504eb9fa67a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:02.117108 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b21f014-423c-47f1-840a-504eb9fa67a6 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:02.128333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1009/1997] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:02.159052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21f014-423c-47f1-840a-504eb9fa67a6 service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:02.171831 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21f014-423c-47f1-840a-504eb9fa67a6 service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:02.177572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c4ebd-26a9-4d1d-859a-a3881e2a0cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:02.178507 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c4ebd-26a9-4d1d-859a-a3881e2a0cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:02.178790 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c4ebd-26a9-4d1d-859a-a3881e2a0cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:02.179064 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c4ebd-26a9-4d1d-859a-a3881e2a0cbc admin admin] PATH_INFO: `/v3/users/badc483a47a8455fb91dabbec8b64831` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:02.180689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c4ebd-26a9-4d1d-859a-a3881e2a0cbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=badc483a47a8455fb91dabbec8b64831)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:02.182914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c4ebd-26a9-4d1d-859a-a3881e2a0cbc admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:02.193644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d06f5-58ec-4eff-b9c2-b0d51c4b0251 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:02.201898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a6828-742e-4574-976b-7fcd0f7f8ee5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:02.202538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a6828-742e-4574-976b-7fcd0f7f8ee5 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:02.202867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a6828-742e-4574-976b-7fcd0f7f8ee5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:02.203421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a6828-742e-4574-976b-7fcd0f7f8ee5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:02.227403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a6828-742e-4574-976b-7fcd0f7f8ee5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:02.228887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a6828-742e-4574-976b-7fcd0f7f8ee5 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:02.235263 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:02.244766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 989/1998] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:02 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:02.269929 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:02.279158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-666c4ebd-26a9-4d1d-859a-a3881e2a0cbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'badc483a47a8455fb91dabbec8b64831'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:02.280717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-666c4ebd-26a9-4d1d-859a-a3881e2a0cbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'badc483a47a8455fb91dabbec8b64831'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:02.307720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-666c4ebd-26a9-4d1d-859a-a3881e2a0cbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'badc483a47a8455fb91dabbec8b64831'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:02.308655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-666c4ebd-26a9-4d1d-859a-a3881e2a0cbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'badc483a47a8455fb91dabbec8b64831'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:02.321389 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-666c4ebd-26a9-4d1d-859a-a3881e2a0cbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'badc483a47a8455fb91dabbec8b64831'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:02.363153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1010/1999] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:02 2022] DELETE /identity/v3/users/badc483a47a8455fb91dabbec8b64831 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:02.368396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442f1ea-248a-444c-b15e-53c0c451754d None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:02.368728 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442f1ea-248a-444c-b15e-53c0c451754d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:02.369078 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442f1ea-248a-444c-b15e-53c0c451754d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:02.410051 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-6442f1ea-248a-444c-b15e-53c0c451754d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:02.414062 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-6442f1ea-248a-444c-b15e-53c0c451754d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:02.444675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c4ebd-26a9-4d1d-859a-a3881e2a0cbc admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:02.446904 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-666c4ebd-26a9-4d1d-859a-a3881e2a0cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:02.538264 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 990/2000] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:02.539391 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442f1ea-248a-444c-b15e-53c0c451754d None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:02.541004 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-6442f1ea-248a-444c-b15e-53c0c451754d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:02.649561 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c4ebd-26a9-4d1d-859a-a3881e2a0cbc admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:02.659596 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442f1ea-248a-444c-b15e-53c0c451754d None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:02.663157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72dc8b-6bd0-465c-879c-5f5092d3e505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:02.664170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72dc8b-6bd0-465c-879c-5f5092d3e505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:02.665208 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72dc8b-6bd0-465c-879c-5f5092d3e505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:02.665702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712aafa-a3ac-4da2-a64d-c97fa85c3644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:02.666332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72dc8b-6bd0-465c-879c-5f5092d3e505 admin admin] PATH_INFO: `/v3/projects/0fe166ee1eb64abfafe330a705e3b03d` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:02.666443 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712aafa-a3ac-4da2-a64d-c97fa85c3644 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:02.666705 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712aafa-a3ac-4da2-a64d-c97fa85c3644 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:02.667082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712aafa-a3ac-4da2-a64d-c97fa85c3644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:02.668644 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72dc8b-6bd0-465c-879c-5f5092d3e505 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fe166ee1eb64abfafe330a705e3b03d)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:02.671770 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72dc8b-6bd0-465c-879c-5f5092d3e505 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:02.690286 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712aafa-a3ac-4da2-a64d-c97fa85c3644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:02.695711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712aafa-a3ac-4da2-a64d-c97fa85c3644 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:02.710814 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 991/2001] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:02.796468 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2f72dc8b-6bd0-465c-879c-5f5092d3e505 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fe166ee1eb64abfafe330a705e3b03d', 'request_id': 'req-2f72dc8b-6bd0-465c-879c-5f5092d3e505'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:02.805614 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2f72dc8b-6bd0-465c-879c-5f5092d3e505 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fe166ee1eb64abfafe330a705e3b03d', 'request_id': 'req-2f72dc8b-6bd0-465c-879c-5f5092d3e505'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:02.815562 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2f72dc8b-6bd0-465c-879c-5f5092d3e505 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fe166ee1eb64abfafe330a705e3b03d', 'request_id': 'req-2f72dc8b-6bd0-465c-879c-5f5092d3e505'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:02.834590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2f72dc8b-6bd0-465c-879c-5f5092d3e505 admin admin] The token cache is being invalidate because project 0fe166ee1eb64abfafe330a705e3b03d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:02.834904 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2f72dc8b-6bd0-465c-879c-5f5092d3e505 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:02.835668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2f72dc8b-6bd0-465c-879c-5f5092d3e505 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:02.837206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1011/2002] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:02 2022] DELETE /identity/v3/projects/0fe166ee1eb64abfafe330a705e3b03d => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:02.841688 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712aafa-a3ac-4da2-a64d-c97fa85c3644 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:02.858783 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712aafa-a3ac-4da2-a64d-c97fa85c3644 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:02.952848 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae5f016-e3d9-4567-aca7-1d7a96d46a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:02.953817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5f016-e3d9-4567-aca7-1d7a96d46a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:02.954147 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5f016-e3d9-4567-aca7-1d7a96d46a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:02.954427 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae5f016-e3d9-4567-aca7-1d7a96d46a5d admin admin] PATH_INFO: `/v3/users/c66ea5585c7b4adcbf5afab7a911d064` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:02.956327 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5f016-e3d9-4567-aca7-1d7a96d46a5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c66ea5585c7b4adcbf5afab7a911d064)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:02.959507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae5f016-e3d9-4567-aca7-1d7a96d46a5d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:03.044534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:03.047909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8ae5f016-e3d9-4567-aca7-1d7a96d46a5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c66ea5585c7b4adcbf5afab7a911d064'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:03.048807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8ae5f016-e3d9-4567-aca7-1d7a96d46a5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c66ea5585c7b4adcbf5afab7a911d064'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:03.078173 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8ae5f016-e3d9-4567-aca7-1d7a96d46a5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c66ea5585c7b4adcbf5afab7a911d064'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:03.091782 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8ae5f016-e3d9-4567-aca7-1d7a96d46a5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c66ea5585c7b4adcbf5afab7a911d064'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:03.110059 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8ae5f016-e3d9-4567-aca7-1d7a96d46a5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c66ea5585c7b4adcbf5afab7a911d064'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:03.127956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 992/2003] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:02 2022] DELETE /identity/v3/users/c66ea5585c7b4adcbf5afab7a911d064 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:03.339648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72dc8b-6bd0-465c-879c-5f5092d3e505 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:03.341330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2f72dc8b-6bd0-465c-879c-5f5092d3e505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:03.378653 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72dc8b-6bd0-465c-879c-5f5092d3e505 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:03.439231 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:03.461672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bd29c-2f02-4753-92a8-8299c6c4287b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:03.462349 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bd29c-2f02-4753-92a8-8299c6c4287b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:03.462627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bd29c-2f02-4753-92a8-8299c6c4287b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:03.462902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bd29c-2f02-4753-92a8-8299c6c4287b admin admin] PATH_INFO: `/v3/projects/12f9a758d17647ebaec796b73ba831f8` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:03.464119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bd29c-2f02-4753-92a8-8299c6c4287b admin admin] RBAC: Authorizing `identity:delete_project(project_id=12f9a758d17647ebaec796b73ba831f8)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:03.466102 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bd29c-2f02-4753-92a8-8299c6c4287b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:03.527937 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-300bd29c-2f02-4753-92a8-8299c6c4287b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12f9a758d17647ebaec796b73ba831f8', 'request_id': 'req-300bd29c-2f02-4753-92a8-8299c6c4287b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:03.535484 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-300bd29c-2f02-4753-92a8-8299c6c4287b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12f9a758d17647ebaec796b73ba831f8', 'request_id': 'req-300bd29c-2f02-4753-92a8-8299c6c4287b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:03.547568 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-300bd29c-2f02-4753-92a8-8299c6c4287b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12f9a758d17647ebaec796b73ba831f8', 'request_id': 'req-300bd29c-2f02-4753-92a8-8299c6c4287b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:03.561496 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-300bd29c-2f02-4753-92a8-8299c6c4287b admin admin] The token cache is being invalidate because project 12f9a758d17647ebaec796b73ba831f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:03.561889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-300bd29c-2f02-4753-92a8-8299c6c4287b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:03.562473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-300bd29c-2f02-4753-92a8-8299c6c4287b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:03.564245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1012/2004] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:03 2022] DELETE /identity/v3/projects/12f9a758d17647ebaec796b73ba831f8 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:03.570023 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5fc1f-cd28-466a-a1f2-1d1aa66a5fc7 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:03.571257 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5fc1f-cd28-466a-a1f2-1d1aa66a5fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:03.571746 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5fc1f-cd28-466a-a1f2-1d1aa66a5fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:03.607944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-36d5fc1f-cd28-466a-a1f2-1d1aa66a5fc7 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:03.609313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-36d5fc1f-cd28-466a-a1f2-1d1aa66a5fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:03.698056 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 993/2005] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:03.706983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dee33f-d51b-43e3-b89a-a2b20b8ab84d None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:03.707265 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dee33f-d51b-43e3-b89a-a2b20b8ab84d None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:03.708098 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dee33f-d51b-43e3-b89a-a2b20b8ab84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:03.742819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-41dee33f-d51b-43e3-b89a-a2b20b8ab84d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:03.744040 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-41dee33f-d51b-43e3-b89a-a2b20b8ab84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:03.757318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1013/2006] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:03.757932 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5fc1f-cd28-466a-a1f2-1d1aa66a5fc7 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:03.774732 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5fc1f-cd28-466a-a1f2-1d1aa66a5fc7 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:03.794962 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06268a-7f94-4e1c-a06d-a1867599e689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:03.795744 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06268a-7f94-4e1c-a06d-a1867599e689 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:03.796091 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06268a-7f94-4e1c-a06d-a1867599e689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:03.796304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b06268a-7f94-4e1c-a06d-a1867599e689 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:03.800146 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06268a-7f94-4e1c-a06d-a1867599e689 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:24:03.800530 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06268a-7f94-4e1c-a06d-a1867599e689 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:03.802834 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b06268a-7f94-4e1c-a06d-a1867599e689 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:03.843775 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 994/2007] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:24:03.846060 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06268a-7f94-4e1c-a06d-a1867599e689 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:03.846635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dee33f-d51b-43e3-b89a-a2b20b8ab84d None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:03.848840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-41dee33f-d51b-43e3-b89a-a2b20b8ab84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:03.858476 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b06268a-7f94-4e1c-a06d-a1867599e689 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:03.870192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112bf818-eec7-48d8-bfc6-ac0601414afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:03.871533 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bf818-eec7-48d8-bfc6-ac0601414afb admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:03.872536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bf818-eec7-48d8-bfc6-ac0601414afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:03.873904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bf818-eec7-48d8-bfc6-ac0601414afb admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:03.876736 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112bf818-eec7-48d8-bfc6-ac0601414afb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:03.880016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112bf818-eec7-48d8-bfc6-ac0601414afb admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:03.890251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dee33f-d51b-43e3-b89a-a2b20b8ab84d None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:03.902074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77782df-8158-4493-b45e-1ce3adcd2633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:03.902779 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77782df-8158-4493-b45e-1ce3adcd2633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:03.902960 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77782df-8158-4493-b45e-1ce3adcd2633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:03.903232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77782df-8158-4493-b45e-1ce3adcd2633 admin admin] PATH_INFO: `/v3/projects/d58cea1eda854e9580a6a3839a45c4a5` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:03.906919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77782df-8158-4493-b45e-1ce3adcd2633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d58cea1eda854e9580a6a3839a45c4a5)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:03.908906 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77782df-8158-4493-b45e-1ce3adcd2633 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:03.932831 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 995/2008] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:03 2022] POST /identity/v3/projects => generated 371 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:03.936760 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112bf818-eec7-48d8-bfc6-ac0601414afb admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:04.010592 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e77782df-8158-4493-b45e-1ce3adcd2633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd58cea1eda854e9580a6a3839a45c4a5', 'request_id': 'req-e77782df-8158-4493-b45e-1ce3adcd2633'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:04.022164 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e77782df-8158-4493-b45e-1ce3adcd2633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd58cea1eda854e9580a6a3839a45c4a5', 'request_id': 'req-e77782df-8158-4493-b45e-1ce3adcd2633'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:04.048608 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e77782df-8158-4493-b45e-1ce3adcd2633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd58cea1eda854e9580a6a3839a45c4a5', 'request_id': 'req-e77782df-8158-4493-b45e-1ce3adcd2633'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:04.071626 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e77782df-8158-4493-b45e-1ce3adcd2633 admin admin] The token cache is being invalidate because project d58cea1eda854e9580a6a3839a45c4a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:04.072124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e77782df-8158-4493-b45e-1ce3adcd2633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:04.072923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e77782df-8158-4493-b45e-1ce3adcd2633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:04.077190 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1014/2009] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:03 2022] DELETE /identity/v3/projects/d58cea1eda854e9580a6a3839a45c4a5 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:04.079786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e829a2-b9a6-432f-8dd9-8291d508e4a2 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:04.080148 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e829a2-b9a6-432f-8dd9-8291d508e4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:04.080515 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e829a2-b9a6-432f-8dd9-8291d508e4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:04.099725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112bf818-eec7-48d8-bfc6-ac0601414afb admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:04.115467 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-94e829a2-b9a6-432f-8dd9-8291d508e4a2 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:04.116752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-94e829a2-b9a6-432f-8dd9-8291d508e4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:04.210914 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:04.263525 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330e6e1e-521f-414d-8dd5-b3280009afb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:04.264175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e6e1e-521f-414d-8dd5-b3280009afb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:04.264456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e6e1e-521f-414d-8dd5-b3280009afb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:04.264725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e6e1e-521f-414d-8dd5-b3280009afb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:04.265421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330e6e1e-521f-414d-8dd5-b3280009afb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:04.268940 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330e6e1e-521f-414d-8dd5-b3280009afb2 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:04.305323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 996/2010] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:03 2022] POST /identity/v3/users => generated 286 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:04.306544 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330e6e1e-521f-414d-8dd5-b3280009afb2 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:04.308781 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-330e6e1e-521f-414d-8dd5-b3280009afb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:04.322431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:04.326336 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330e6e1e-521f-414d-8dd5-b3280009afb2 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:04.331166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cfbbe-34e3-47e7-a095-1f09bfc1430c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:04.332313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cfbbe-34e3-47e7-a095-1f09bfc1430c admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:04.332708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cfbbe-34e3-47e7-a095-1f09bfc1430c admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:04.333500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cfbbe-34e3-47e7-a095-1f09bfc1430c admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:04.334393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cfbbe-34e3-47e7-a095-1f09bfc1430c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:04.336230 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cfbbe-34e3-47e7-a095-1f09bfc1430c admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:04.350346 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 997/2011] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:24:04.353832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cfbbe-34e3-47e7-a095-1f09bfc1430c admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:04.365177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1015/2012] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:04.365717 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cfbbe-34e3-47e7-a095-1f09bfc1430c admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:04.369170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf70801-f25a-4d70-b0af-24a05fe107b6 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:04.369448 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf70801-f25a-4d70-b0af-24a05fe107b6 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:04.369722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf70801-f25a-4d70-b0af-24a05fe107b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:04.378033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62ca74-b946-4d6d-b374-bb0dbbb74df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:04.378564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62ca74-b946-4d6d-b374-bb0dbbb74df1 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:04.378842 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62ca74-b946-4d6d-b374-bb0dbbb74df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:04.379115 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62ca74-b946-4d6d-b374-bb0dbbb74df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:04.380384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62ca74-b946-4d6d-b374-bb0dbbb74df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:04.383606 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62ca74-b946-4d6d-b374-bb0dbbb74df1 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:04.407711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 998/2013] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:24:04.412877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62ca74-b946-4d6d-b374-bb0dbbb74df1 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:04.422281 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-adf70801-f25a-4d70-b0af-24a05fe107b6 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:04.424378 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-adf70801-f25a-4d70-b0af-24a05fe107b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:04.432675 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62ca74-b946-4d6d-b374-bb0dbbb74df1 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:04.440117 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a480986-a2e2-422d-8e2c-d1abee8dafe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:04.442833 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a480986-a2e2-422d-8e2c-d1abee8dafe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:04.443352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a480986-a2e2-422d-8e2c-d1abee8dafe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:04.444882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a480986-a2e2-422d-8e2c-d1abee8dafe4 admin admin] PATH_INFO: `/v3/projects/1f563e7675ef4d758e09e1a78b891c9d/users/49280c781ba745aa83bf4c5db9359973/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:04.446412 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1016/2014] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:04.450489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf70801-f25a-4d70-b0af-24a05fe107b6 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:04.479121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a480986-a2e2-422d-8e2c-d1abee8dafe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f563e7675ef4d758e09e1a78b891c9d, user_id=49280c781ba745aa83bf4c5db9359973, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:04.484251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a480986-a2e2-422d-8e2c-d1abee8dafe4 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:04.497012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf70801-f25a-4d70-b0af-24a05fe107b6 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:04.505908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 999/2015] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:04 2022] PUT /identity/v3/projects/1f563e7675ef4d758e09e1a78b891c9d/users/49280c781ba745aa83bf4c5db9359973/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:04.511059 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c31087-9270-4edf-9167-6c2cfe92a0c0 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:04.511588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c31087-9270-4edf-9167-6c2cfe92a0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:04.512849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c31087-9270-4edf-9167-6c2cfe92a0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:04.541452 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-42c31087-9270-4edf-9167-6c2cfe92a0c0 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:04.543459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-42c31087-9270-4edf-9167-6c2cfe92a0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:04.659629 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118660d4-1b15-43c4-b5d6-11fb46841d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:04.661985 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118660d4-1b15-43c4-b5d6-11fb46841d11 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:04.662271 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118660d4-1b15-43c4-b5d6-11fb46841d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:04.662598 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118660d4-1b15-43c4-b5d6-11fb46841d11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:04.663203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1000/2016] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:04.663678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118660d4-1b15-43c4-b5d6-11fb46841d11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:24:04.663678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118660d4-1b15-43c4-b5d6-11fb46841d11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:04.665280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118660d4-1b15-43c4-b5d6-11fb46841d11 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:04.675645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c31087-9270-4edf-9167-6c2cfe92a0c0 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:04.679878 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-42c31087-9270-4edf-9167-6c2cfe92a0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:04.688685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1017/2017] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:24:04.693990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118660d4-1b15-43c4-b5d6-11fb46841d11 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:04.712986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118660d4-1b15-43c4-b5d6-11fb46841d11 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:04.729526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b647f4-97fb-406a-801e-69ea3872bef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:04.730876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b647f4-97fb-406a-801e-69ea3872bef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:04.731177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b647f4-97fb-406a-801e-69ea3872bef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:04.731471 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b647f4-97fb-406a-801e-69ea3872bef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:04.732109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b647f4-97fb-406a-801e-69ea3872bef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:04.736468 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b647f4-97fb-406a-801e-69ea3872bef0 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:04.798510 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c31087-9270-4edf-9167-6c2cfe92a0c0 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:04.806722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5cbfb-a665-4c4a-8816-c2aef1bcc211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:04.807484 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5cbfb-a665-4c4a-8816-c2aef1bcc211 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:04.808249 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5cbfb-a665-4c4a-8816-c2aef1bcc211 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:04.808677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5cbfb-a665-4c4a-8816-c2aef1bcc211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:04.830380 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5cbfb-a665-4c4a-8816-c2aef1bcc211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:04.833859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5cbfb-a665-4c4a-8816-c2aef1bcc211 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:04.836841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1018/2018] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:04 2022] POST /identity/v3/projects => generated 387 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:04.841384 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b647f4-97fb-406a-801e-69ea3872bef0 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:04.872198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1001/2019] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:24:04.998662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b647f4-97fb-406a-801e-69ea3872bef0 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:05.032326 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8bd0b-4d26-470f-a894-9a716995a696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:05.033058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8bd0b-4d26-470f-a894-9a716995a696 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:05.033372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8bd0b-4d26-470f-a894-9a716995a696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:05.033647 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8bd0b-4d26-470f-a894-9a716995a696 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:05.035424 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8bd0b-4d26-470f-a894-9a716995a696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:05.040674 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8bd0b-4d26-470f-a894-9a716995a696 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:05.106991 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1019/2020] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:04 2022] POST /identity/v3/users => generated 294 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:05.111394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5cbfb-a665-4c4a-8816-c2aef1bcc211 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:05.132926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5cbfb-a665-4c4a-8816-c2aef1bcc211 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:05.138582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6cb3b4-0926-4c9a-a7f1-943a5788ca2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:05.139374 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cb3b4-0926-4c9a-a7f1-943a5788ca2a admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:05.139878 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cb3b4-0926-4c9a-a7f1-943a5788ca2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:05.140250 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cb3b4-0926-4c9a-a7f1-943a5788ca2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:05.142971 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6cb3b4-0926-4c9a-a7f1-943a5788ca2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:05.144888 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6cb3b4-0926-4c9a-a7f1-943a5788ca2a admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:05.161648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8bd0b-4d26-470f-a894-9a716995a696 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:05.162191 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1002/2021] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:05.177287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8bd0b-4d26-470f-a894-9a716995a696 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:05.190043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9098dd5-6acd-408d-96e4-f5f32f8b1c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:05.193327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9098dd5-6acd-408d-96e4-f5f32f8b1c7d admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:05.194589 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9098dd5-6acd-408d-96e4-f5f32f8b1c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:05.195024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9098dd5-6acd-408d-96e4-f5f32f8b1c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:05.195706 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9098dd5-6acd-408d-96e4-f5f32f8b1c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:05.199588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9098dd5-6acd-408d-96e4-f5f32f8b1c7d admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:05.228182 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1020/2022] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:05.233278 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6cb3b4-0926-4c9a-a7f1-943a5788ca2a admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:05.250152 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6cb3b4-0926-4c9a-a7f1-943a5788ca2a admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:05.265919 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608aad16-7eae-4700-af43-63847d928203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:05.266768 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608aad16-7eae-4700-af43-63847d928203 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:05.267154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608aad16-7eae-4700-af43-63847d928203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:05.267694 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608aad16-7eae-4700-af43-63847d928203 admin admin] PATH_INFO: `/v3/projects/e728aab6b16448df8a492b5138872e40/users/f2dd322872da48dc81d9e39948596e53/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:05.295156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:05.313432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608aad16-7eae-4700-af43-63847d928203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e728aab6b16448df8a492b5138872e40, user_id=f2dd322872da48dc81d9e39948596e53, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:05.323903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608aad16-7eae-4700-af43-63847d928203 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:05.355858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1003/2023] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:05 2022] PUT /identity/v3/projects/e728aab6b16448df8a492b5138872e40/users/f2dd322872da48dc81d9e39948596e53/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:05.368189 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58551cce-59da-4e8e-8350-df2395689d4b None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:05.369056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58551cce-59da-4e8e-8350-df2395689d4b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:05.369639 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58551cce-59da-4e8e-8350-df2395689d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:05.436705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-58551cce-59da-4e8e-8350-df2395689d4b None None] MFA Rules not processed for user `f2dd322872da48dc81d9e39948596e53`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:05.440473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-58551cce-59da-4e8e-8350-df2395689d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:05.506638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:05.630775 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608aad16-7eae-4700-af43-63847d928203 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:05.632270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1021/2024] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:05.784017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608aad16-7eae-4700-af43-63847d928203 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:05.791640 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172552ca-d953-48d3-a34e-a8d8708aa0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:05.792615 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172552ca-d953-48d3-a34e-a8d8708aa0bc admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:05.793259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172552ca-d953-48d3-a34e-a8d8708aa0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:05.793742 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172552ca-d953-48d3-a34e-a8d8708aa0bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:05.794618 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172552ca-d953-48d3-a34e-a8d8708aa0bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:05.796862 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172552ca-d953-48d3-a34e-a8d8708aa0bc admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:05.853186 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1004/2025] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:05 2022] POST /identity/v3/projects => generated 387 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:05.858255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58551cce-59da-4e8e-8350-df2395689d4b None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:05.875882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58551cce-59da-4e8e-8350-df2395689d4b None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:05.960854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e5c98-c2c4-4b3f-8f06-80a16af7cdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:05.961508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e5c98-c2c4-4b3f-8f06-80a16af7cdb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:05.961786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e5c98-c2c4-4b3f-8f06-80a16af7cdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:05.962054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e5c98-c2c4-4b3f-8f06-80a16af7cdb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:05.962676 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e5c98-c2c4-4b3f-8f06-80a16af7cdb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:05.964572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e5c98-c2c4-4b3f-8f06-80a16af7cdb5 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:05.990318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1022/2026] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:05 2022] POST /identity/v3/users => generated 294 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:05.994440 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172552ca-d953-48d3-a34e-a8d8708aa0bc admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:06.005169 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172552ca-d953-48d3-a34e-a8d8708aa0bc admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:06.009825 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfc02f-d240-4553-9013-cc49b3be5ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:06.010434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfc02f-d240-4553-9013-cc49b3be5ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:06.010712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfc02f-d240-4553-9013-cc49b3be5ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:06.011039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfc02f-d240-4553-9013-cc49b3be5ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:06.011684 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bfc02f-d240-4553-9013-cc49b3be5ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:06.013355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bfc02f-d240-4553-9013-cc49b3be5ef9 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:06.031089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1005/2027] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:06.032182 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e5c98-c2c4-4b3f-8f06-80a16af7cdb5 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:06.044656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e5c98-c2c4-4b3f-8f06-80a16af7cdb5 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:06.049793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481edcd5-f931-4e7a-a03d-4fa7ed96bc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:06.050404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481edcd5-f931-4e7a-a03d-4fa7ed96bc88 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:06.050746 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481edcd5-f931-4e7a-a03d-4fa7ed96bc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:06.051026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481edcd5-f931-4e7a-a03d-4fa7ed96bc88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:06.051715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481edcd5-f931-4e7a-a03d-4fa7ed96bc88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:06.053461 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481edcd5-f931-4e7a-a03d-4fa7ed96bc88 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:06.065181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1023/2028] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:06.070137 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfc02f-d240-4553-9013-cc49b3be5ef9 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:06.084376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfc02f-d240-4553-9013-cc49b3be5ef9 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:06.090075 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39af82a-c80e-406e-9a65-76335ee5b970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:06.091119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39af82a-c80e-406e-9a65-76335ee5b970 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:06.091598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39af82a-c80e-406e-9a65-76335ee5b970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:06.092141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39af82a-c80e-406e-9a65-76335ee5b970 admin admin] PATH_INFO: `/v3/projects/ce3a5d2f00c4404784a72a106dd0e17f/users/82d287ba18b146809f4727597e93a1bb/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:06.132994 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39af82a-c80e-406e-9a65-76335ee5b970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce3a5d2f00c4404784a72a106dd0e17f, user_id=82d287ba18b146809f4727597e93a1bb, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:06.136967 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39af82a-c80e-406e-9a65-76335ee5b970 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:06.165583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1006/2029] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:06 2022] PUT /identity/v3/projects/ce3a5d2f00c4404784a72a106dd0e17f/users/82d287ba18b146809f4727597e93a1bb/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:06.175680 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d293814-7e60-4eb8-bbe7-3a0f5facedfe None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:06.176063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d293814-7e60-4eb8-bbe7-3a0f5facedfe None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:06.176847 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d293814-7e60-4eb8-bbe7-3a0f5facedfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:06.224304 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-6d293814-7e60-4eb8-bbe7-3a0f5facedfe None None] MFA Rules not processed for user `82d287ba18b146809f4727597e93a1bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:06.225820 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-6d293814-7e60-4eb8-bbe7-3a0f5facedfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:06.314596 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1024/2030] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:06.321933 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39af82a-c80e-406e-9a65-76335ee5b970 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:06.403527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39af82a-c80e-406e-9a65-76335ee5b970 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:06.408417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23717023-1f38-4f48-854e-d46d0e34fc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:06.409038 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23717023-1f38-4f48-854e-d46d0e34fc85 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:06.409357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23717023-1f38-4f48-854e-d46d0e34fc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:06.409633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23717023-1f38-4f48-854e-d46d0e34fc85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:06.410272 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23717023-1f38-4f48-854e-d46d0e34fc85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:06.413062 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23717023-1f38-4f48-854e-d46d0e34fc85 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:06.449366 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1007/2031] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:06 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:06.452691 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d293814-7e60-4eb8-bbe7-3a0f5facedfe None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:06.464604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d293814-7e60-4eb8-bbe7-3a0f5facedfe None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:06.503226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:06.533284 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d5568-97e7-463a-9003-369a57b01289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:06.533929 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d5568-97e7-463a-9003-369a57b01289 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:06.534209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d5568-97e7-463a-9003-369a57b01289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:06.534477 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d5568-97e7-463a-9003-369a57b01289 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:06.535099 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d5568-97e7-463a-9003-369a57b01289 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:06.537232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d5568-97e7-463a-9003-369a57b01289 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:06.563572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1025/2032] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:06 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:06.565385 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23717023-1f38-4f48-854e-d46d0e34fc85 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:06.577095 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23717023-1f38-4f48-854e-d46d0e34fc85 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:06.583163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e0723-f72b-403c-8ca4-65bdfcbfe8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:06.583838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e0723-f72b-403c-8ca4-65bdfcbfe8f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:06.584115 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e0723-f72b-403c-8ca4-65bdfcbfe8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:06.584437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e0723-f72b-403c-8ca4-65bdfcbfe8f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:06.585087 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e0723-f72b-403c-8ca4-65bdfcbfe8f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:06.587096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e0723-f72b-403c-8ca4-65bdfcbfe8f2 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:06.603396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1008/2033] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:06.605006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d5568-97e7-463a-9003-369a57b01289 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:06.616931 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d5568-97e7-463a-9003-369a57b01289 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:06.621684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de8892-d8bf-41b6-b3ab-21f8f1b599aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:06.622345 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de8892-d8bf-41b6-b3ab-21f8f1b599aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:06.622620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de8892-d8bf-41b6-b3ab-21f8f1b599aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:06.622890 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de8892-d8bf-41b6-b3ab-21f8f1b599aa admin admin] PATH_INFO: `/v3/projects/8920549484404e778283415dd9fef34f/users/c388ff177cc7407cbdd6b4cfad510b1f/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:06.641986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de8892-d8bf-41b6-b3ab-21f8f1b599aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8920549484404e778283415dd9fef34f, user_id=c388ff177cc7407cbdd6b4cfad510b1f, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:06.645776 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de8892-d8bf-41b6-b3ab-21f8f1b599aa admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:06.670089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1026/2034] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:06 2022] PUT /identity/v3/projects/8920549484404e778283415dd9fef34f/users/c388ff177cc7407cbdd6b4cfad510b1f/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:06.671336 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e0723-f72b-403c-8ca4-65bdfcbfe8f2 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:06.682782 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e0723-f72b-403c-8ca4-65bdfcbfe8f2 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:06.691399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:06.750784 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df61acb-c24a-4e4e-9543-598910159d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:06.751754 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df61acb-c24a-4e4e-9543-598910159d30 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:06.752224 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df61acb-c24a-4e4e-9543-598910159d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:06.752692 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df61acb-c24a-4e4e-9543-598910159d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:06.753396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df61acb-c24a-4e4e-9543-598910159d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:06.755573 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df61acb-c24a-4e4e-9543-598910159d30 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:06.768387 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1009/2035] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:06.773582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de8892-d8bf-41b6-b3ab-21f8f1b599aa admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:06.785323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de8892-d8bf-41b6-b3ab-21f8f1b599aa admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:06.791016 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89637216-2c32-4eb2-8775-c0de69ed0617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:06.791683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89637216-2c32-4eb2-8775-c0de69ed0617 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:06.791964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89637216-2c32-4eb2-8775-c0de69ed0617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:06.792240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89637216-2c32-4eb2-8775-c0de69ed0617 admin admin] PATH_INFO: `/v3/projects/8920549484404e778283415dd9fef34f/users/c388ff177cc7407cbdd6b4cfad510b1f/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:06.794195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89637216-2c32-4eb2-8775-c0de69ed0617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8920549484404e778283415dd9fef34f, user_id=c388ff177cc7407cbdd6b4cfad510b1f, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:06.796815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89637216-2c32-4eb2-8775-c0de69ed0617 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:06.808839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1027/2036] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:06 2022] PUT /identity/v3/projects/8920549484404e778283415dd9fef34f/users/c388ff177cc7407cbdd6b4cfad510b1f/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:06.823209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83cf8a-b2a6-4844-a296-fa9930c4539c None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:06.823696 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83cf8a-b2a6-4844-a296-fa9930c4539c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:06.824485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83cf8a-b2a6-4844-a296-fa9930c4539c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:06.888904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-7a83cf8a-b2a6-4844-a296-fa9930c4539c None None] MFA Rules not processed for user `c388ff177cc7407cbdd6b4cfad510b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:06.891002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7a83cf8a-b2a6-4844-a296-fa9930c4539c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:07.011035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1010/2037] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:06 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:07.031973 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89637216-2c32-4eb2-8775-c0de69ed0617 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:07.033761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-89637216-2c32-4eb2-8775-c0de69ed0617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:07.115408 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89637216-2c32-4eb2-8775-c0de69ed0617 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:07.121353 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd892d76-8fb0-4d3c-8f41-b6e7432de318 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:07.121999 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd892d76-8fb0-4d3c-8f41-b6e7432de318 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:07.122279 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd892d76-8fb0-4d3c-8f41-b6e7432de318 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:07.122560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd892d76-8fb0-4d3c-8f41-b6e7432de318 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:07.134594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd892d76-8fb0-4d3c-8f41-b6e7432de318 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:07.136196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd892d76-8fb0-4d3c-8f41-b6e7432de318 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:07.157514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1028/2038] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:07.836533 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83cf8a-b2a6-4844-a296-fa9930c4539c None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:07.838038 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7a83cf8a-b2a6-4844-a296-fa9930c4539c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:07.852026 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83cf8a-b2a6-4844-a296-fa9930c4539c None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:07.855853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb4f19-118d-420f-a3f9-468195147d83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:07.856566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb4f19-118d-420f-a3f9-468195147d83 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:07.856925 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb4f19-118d-420f-a3f9-468195147d83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:07.857307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb4f19-118d-420f-a3f9-468195147d83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:07.927862 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb4f19-118d-420f-a3f9-468195147d83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:07.929354 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb4f19-118d-420f-a3f9-468195147d83 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:07.939668 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1011/2039] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:10.185193 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7874fa-8736-47d8-b538-744e15f4a5d0 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:10.185646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7874fa-8736-47d8-b538-744e15f4a5d0 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:10.185770 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7874fa-8736-47d8-b538-744e15f4a5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:10.222052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-1a7874fa-8736-47d8-b538-744e15f4a5d0 None None] MFA Rules not processed for user `49280c781ba745aa83bf4c5db9359973`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:10.223682 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-1a7874fa-8736-47d8-b538-744e15f4a5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:10.324544 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1029/2040] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:10.329957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb4f19-118d-420f-a3f9-468195147d83 service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:10.394635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:10.440536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb4f19-118d-420f-a3f9-468195147d83 service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:10.448963 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326e787-34da-4e2d-9b9b-3648957ff875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:10.449729 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326e787-34da-4e2d-9b9b-3648957ff875 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:10.450068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326e787-34da-4e2d-9b9b-3648957ff875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:10.450357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326e787-34da-4e2d-9b9b-3648957ff875 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:10.451464 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326e787-34da-4e2d-9b9b-3648957ff875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:10.456423 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326e787-34da-4e2d-9b9b-3648957ff875 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:10.499915 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1012/2041] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:10 2022] POST /identity/v3/projects => generated 371 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:10.505397 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7874fa-8736-47d8-b538-744e15f4a5d0 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:10.520713 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7874fa-8736-47d8-b538-744e15f4a5d0 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:10.553581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:10.615418 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a966564-911e-4435-94cf-5899918dd30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:10.616043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a966564-911e-4435-94cf-5899918dd30e admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:10.616318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a966564-911e-4435-94cf-5899918dd30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:10.616593 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a966564-911e-4435-94cf-5899918dd30e admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:10.617254 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a966564-911e-4435-94cf-5899918dd30e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:10.619193 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a966564-911e-4435-94cf-5899918dd30e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:10.645174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1030/2042] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:10 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:10.648792 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326e787-34da-4e2d-9b9b-3648957ff875 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:10.661659 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326e787-34da-4e2d-9b9b-3648957ff875 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:10.666423 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5fd6da-69fb-40c4-b6d3-de9f39d9cb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:10.667151 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fd6da-69fb-40c4-b6d3-de9f39d9cb75 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:10.667436 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fd6da-69fb-40c4-b6d3-de9f39d9cb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:10.667717 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fd6da-69fb-40c4-b6d3-de9f39d9cb75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:10.668369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5fd6da-69fb-40c4-b6d3-de9f39d9cb75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:10.670002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5fd6da-69fb-40c4-b6d3-de9f39d9cb75 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:10.685101 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1013/2043] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:10.686652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a966564-911e-4435-94cf-5899918dd30e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:10.697572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a966564-911e-4435-94cf-5899918dd30e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:10.702945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc23401-186a-471b-9c45-d2d7c56a56cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:10.704763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc23401-186a-471b-9c45-d2d7c56a56cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:10.705232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc23401-186a-471b-9c45-d2d7c56a56cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:10.705521 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc23401-186a-471b-9c45-d2d7c56a56cb admin admin] PATH_INFO: `/v3/projects/f2df1492593c40429906ed174145edb1/users/93fb252cab314529a97fb513e8fa28b5/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:10.728378 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc23401-186a-471b-9c45-d2d7c56a56cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2df1492593c40429906ed174145edb1, user_id=93fb252cab314529a97fb513e8fa28b5, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:10.731080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc23401-186a-471b-9c45-d2d7c56a56cb admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:10.744688 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1031/2044] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:10 2022] PUT /identity/v3/projects/f2df1492593c40429906ed174145edb1/users/93fb252cab314529a97fb513e8fa28b5/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:10.748764 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5fd6da-69fb-40c4-b6d3-de9f39d9cb75 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:10.759316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5fd6da-69fb-40c4-b6d3-de9f39d9cb75 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:10.817973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfc334-72e3-4551-a3de-42d9eb8000d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:10.818514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfc334-72e3-4551-a3de-42d9eb8000d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:10.818817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfc334-72e3-4551-a3de-42d9eb8000d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:10.819072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfc334-72e3-4551-a3de-42d9eb8000d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:10.819710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bfc334-72e3-4551-a3de-42d9eb8000d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:10.821339 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bfc334-72e3-4551-a3de-42d9eb8000d5 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:10.830945 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1014/2045] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:10.835356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc23401-186a-471b-9c45-d2d7c56a56cb admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:10.845534 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc23401-186a-471b-9c45-d2d7c56a56cb admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:10.850634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca7281-41a8-49e3-a8ed-5d40107095b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:10.851282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca7281-41a8-49e3-a8ed-5d40107095b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:10.851562 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca7281-41a8-49e3-a8ed-5d40107095b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:10.851833 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca7281-41a8-49e3-a8ed-5d40107095b1 admin admin] PATH_INFO: `/v3/projects/f2df1492593c40429906ed174145edb1/users/93fb252cab314529a97fb513e8fa28b5/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:10.853722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca7281-41a8-49e3-a8ed-5d40107095b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2df1492593c40429906ed174145edb1, user_id=93fb252cab314529a97fb513e8fa28b5, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:10.856521 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca7281-41a8-49e3-a8ed-5d40107095b1 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:10.867271 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1032/2046] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:10 2022] PUT /identity/v3/projects/f2df1492593c40429906ed174145edb1/users/93fb252cab314529a97fb513e8fa28b5/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:12.725039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfc334-72e3-4551-a3de-42d9eb8000d5 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:12.726900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b9bfc334-72e3-4551-a3de-42d9eb8000d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:12.740020 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfc334-72e3-4551-a3de-42d9eb8000d5 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:12.859335 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59afa70-d7cc-4ddd-a7b2-4aed61ee1c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:12.860123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59afa70-d7cc-4ddd-a7b2-4aed61ee1c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:12.860386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59afa70-d7cc-4ddd-a7b2-4aed61ee1c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:12.861388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59afa70-d7cc-4ddd-a7b2-4aed61ee1c8d admin admin] PATH_INFO: `/v3/users/680e2b80bbdd4b319eb4bb5faa1fb544` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:12.862725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59afa70-d7cc-4ddd-a7b2-4aed61ee1c8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=680e2b80bbdd4b319eb4bb5faa1fb544)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:12.866428 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59afa70-d7cc-4ddd-a7b2-4aed61ee1c8d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:12.936549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:12.940007 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-c59afa70-d7cc-4ddd-a7b2-4aed61ee1c8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '680e2b80bbdd4b319eb4bb5faa1fb544'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:12.940923 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-c59afa70-d7cc-4ddd-a7b2-4aed61ee1c8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '680e2b80bbdd4b319eb4bb5faa1fb544'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:12.959586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-c59afa70-d7cc-4ddd-a7b2-4aed61ee1c8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '680e2b80bbdd4b319eb4bb5faa1fb544'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:12.968798 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-c59afa70-d7cc-4ddd-a7b2-4aed61ee1c8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '680e2b80bbdd4b319eb4bb5faa1fb544'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:12.969716 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-c59afa70-d7cc-4ddd-a7b2-4aed61ee1c8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '680e2b80bbdd4b319eb4bb5faa1fb544'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:12.985780 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1015/2047] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:12 2022] DELETE /identity/v3/users/680e2b80bbdd4b319eb4bb5faa1fb544 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:24:12.990742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b6793-3faa-4010-a7a4-a68089b52530 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:12.991257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b6793-3faa-4010-a7a4-a68089b52530 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:12.991364 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b6793-3faa-4010-a7a4-a68089b52530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:13.012830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-4a4b6793-3faa-4010-a7a4-a68089b52530 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:13.014079 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-4a4b6793-3faa-4010-a7a4-a68089b52530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:13.104734 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1033/2048] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:13.116789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59afa70-d7cc-4ddd-a7b2-4aed61ee1c8d admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:13.119883 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c59afa70-d7cc-4ddd-a7b2-4aed61ee1c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:13.218468 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59afa70-d7cc-4ddd-a7b2-4aed61ee1c8d admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:13.223395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec5be3-94d7-4a91-a4ce-f41147a58b31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:13.224578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec5be3-94d7-4a91-a4ce-f41147a58b31 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:13.224578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec5be3-94d7-4a91-a4ce-f41147a58b31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:13.224904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec5be3-94d7-4a91-a4ce-f41147a58b31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:13.239567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec5be3-94d7-4a91-a4ce-f41147a58b31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:13.241831 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec5be3-94d7-4a91-a4ce-f41147a58b31 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:13.253650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1016/2049] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:13.733027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b6793-3faa-4010-a7a4-a68089b52530 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:13.734421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-4a4b6793-3faa-4010-a7a4-a68089b52530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:13.762098 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b6793-3faa-4010-a7a4-a68089b52530 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:13.779472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4c623-b78f-40d6-8b87-5bc1de362aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:13.780410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4c623-b78f-40d6-8b87-5bc1de362aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:13.783290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4c623-b78f-40d6-8b87-5bc1de362aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:13.784948 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4c623-b78f-40d6-8b87-5bc1de362aa3 admin admin] PATH_INFO: `/v3/projects/0dc9b3c68d774159a5acb1b15837da13` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:13.786689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4c623-b78f-40d6-8b87-5bc1de362aa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dc9b3c68d774159a5acb1b15837da13)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:13.788722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4c623-b78f-40d6-8b87-5bc1de362aa3 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:13.874911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-94a4c623-b78f-40d6-8b87-5bc1de362aa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dc9b3c68d774159a5acb1b15837da13', 'request_id': 'req-94a4c623-b78f-40d6-8b87-5bc1de362aa3'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:13.898873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-94a4c623-b78f-40d6-8b87-5bc1de362aa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dc9b3c68d774159a5acb1b15837da13', 'request_id': 'req-94a4c623-b78f-40d6-8b87-5bc1de362aa3'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:13.919886 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-94a4c623-b78f-40d6-8b87-5bc1de362aa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dc9b3c68d774159a5acb1b15837da13', 'request_id': 'req-94a4c623-b78f-40d6-8b87-5bc1de362aa3'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:13.931825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:13.940371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-94a4c623-b78f-40d6-8b87-5bc1de362aa3 admin admin] The token cache is being invalidate because project 0dc9b3c68d774159a5acb1b15837da13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:13.940869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-94a4c623-b78f-40d6-8b87-5bc1de362aa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:13.942084 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-94a4c623-b78f-40d6-8b87-5bc1de362aa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:13.945453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1034/2050] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:13 2022] DELETE /identity/v3/projects/0dc9b3c68d774159a5acb1b15837da13 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:13.949339 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec5be3-94d7-4a91-a4ce-f41147a58b31 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:13.965009 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec5be3-94d7-4a91-a4ce-f41147a58b31 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:14.080599 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312abdca-f1f9-4b8a-8796-03859ca0b908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:14.081650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312abdca-f1f9-4b8a-8796-03859ca0b908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:14.083416 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312abdca-f1f9-4b8a-8796-03859ca0b908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:14.083850 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312abdca-f1f9-4b8a-8796-03859ca0b908 admin admin] PATH_INFO: `/v3/users/8ef090fb80104395916889a91c717102` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:14.085340 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312abdca-f1f9-4b8a-8796-03859ca0b908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ef090fb80104395916889a91c717102)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:14.087460 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312abdca-f1f9-4b8a-8796-03859ca0b908 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:14.202628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-312abdca-f1f9-4b8a-8796-03859ca0b908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ef090fb80104395916889a91c717102'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:14.204269 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-312abdca-f1f9-4b8a-8796-03859ca0b908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ef090fb80104395916889a91c717102'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:14.226251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-312abdca-f1f9-4b8a-8796-03859ca0b908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ef090fb80104395916889a91c717102'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:14.231757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:14.245978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-312abdca-f1f9-4b8a-8796-03859ca0b908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ef090fb80104395916889a91c717102'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:14.247997 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-312abdca-f1f9-4b8a-8796-03859ca0b908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ef090fb80104395916889a91c717102'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:14.295257 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1017/2051] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:13 2022] DELETE /identity/v3/users/8ef090fb80104395916889a91c717102 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:14.990654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4c623-b78f-40d6-8b87-5bc1de362aa3 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:14.992359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-94a4c623-b78f-40d6-8b87-5bc1de362aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:15.030293 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4c623-b78f-40d6-8b87-5bc1de362aa3 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:15.209119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0d855-ee9f-4bac-b341-7728815e3660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:15.212628 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0d855-ee9f-4bac-b341-7728815e3660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:15.213253 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0d855-ee9f-4bac-b341-7728815e3660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:15.213687 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0d855-ee9f-4bac-b341-7728815e3660 admin admin] PATH_INFO: `/v3/projects/0b143975e7064ff5b962956bc1014668` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:15.215344 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0d855-ee9f-4bac-b341-7728815e3660 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b143975e7064ff5b962956bc1014668)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:15.217559 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0d855-ee9f-4bac-b341-7728815e3660 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:15.304818 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5eb0d855-ee9f-4bac-b341-7728815e3660 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b143975e7064ff5b962956bc1014668', 'request_id': 'req-5eb0d855-ee9f-4bac-b341-7728815e3660'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:15.319384 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5eb0d855-ee9f-4bac-b341-7728815e3660 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b143975e7064ff5b962956bc1014668', 'request_id': 'req-5eb0d855-ee9f-4bac-b341-7728815e3660'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:15.339715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5eb0d855-ee9f-4bac-b341-7728815e3660 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b143975e7064ff5b962956bc1014668', 'request_id': 'req-5eb0d855-ee9f-4bac-b341-7728815e3660'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:15.366633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5eb0d855-ee9f-4bac-b341-7728815e3660 admin admin] The token cache is being invalidate because project 0b143975e7064ff5b962956bc1014668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:15.366633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5eb0d855-ee9f-4bac-b341-7728815e3660 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:15.366633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5eb0d855-ee9f-4bac-b341-7728815e3660 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:15.366633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1035/2052] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:14 2022] DELETE /identity/v3/projects/0b143975e7064ff5b962956bc1014668 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:15.379352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147d26a-fe60-4c5e-860a-7103ca890143 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:15.380183 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147d26a-fe60-4c5e-860a-7103ca890143 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:15.380605 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147d26a-fe60-4c5e-860a-7103ca890143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:15.427487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-d147d26a-fe60-4c5e-860a-7103ca890143 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:15.432657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d147d26a-fe60-4c5e-860a-7103ca890143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:15.532780 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1018/2053] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:15.539633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc63e9-1478-4bff-bc3d-7dafee6fe8c9 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:15.540142 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc63e9-1478-4bff-bc3d-7dafee6fe8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:15.540590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc63e9-1478-4bff-bc3d-7dafee6fe8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:15.587737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-efcc63e9-1478-4bff-bc3d-7dafee6fe8c9 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:15.589372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-efcc63e9-1478-4bff-bc3d-7dafee6fe8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:15.610217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1036/2054] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:15.615223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147d26a-fe60-4c5e-860a-7103ca890143 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:15.629873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147d26a-fe60-4c5e-860a-7103ca890143 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:15.636659 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1e7c7-1f30-4c37-8dcf-b8e088deaf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:15.637421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1e7c7-1f30-4c37-8dcf-b8e088deaf65 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:15.637710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1e7c7-1f30-4c37-8dcf-b8e088deaf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:15.638190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1e7c7-1f30-4c37-8dcf-b8e088deaf65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:15.638873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee1e7c7-1f30-4c37-8dcf-b8e088deaf65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:24:15.639182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee1e7c7-1f30-4c37-8dcf-b8e088deaf65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:15.641198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee1e7c7-1f30-4c37-8dcf-b8e088deaf65 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:15.658813 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1019/2055] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:24:15.662835 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc63e9-1478-4bff-bc3d-7dafee6fe8c9 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:15.672814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc63e9-1478-4bff-bc3d-7dafee6fe8c9 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:15.688993 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d7bb1-9101-4cda-9f51-1c3c2b4e0d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:15.689785 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d7bb1-9101-4cda-9f51-1c3c2b4e0d2c admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:15.690132 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d7bb1-9101-4cda-9f51-1c3c2b4e0d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:15.690475 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d7bb1-9101-4cda-9f51-1c3c2b4e0d2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:15.692459 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d7bb1-9101-4cda-9f51-1c3c2b4e0d2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:15.693990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d7bb1-9101-4cda-9f51-1c3c2b4e0d2c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:15.742043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1037/2056] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:15 2022] POST /identity/v3/projects => generated 401 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:15.746037 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1e7c7-1f30-4c37-8dcf-b8e088deaf65 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:15.759713 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1e7c7-1f30-4c37-8dcf-b8e088deaf65 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:15.882257 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeff1f5-dcb0-4d80-9d46-a9fdf52ba5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:15.885986 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeff1f5-dcb0-4d80-9d46-a9fdf52ba5bd admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:15.886533 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeff1f5-dcb0-4d80-9d46-a9fdf52ba5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:15.887517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeff1f5-dcb0-4d80-9d46-a9fdf52ba5bd admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:15.889592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeff1f5-dcb0-4d80-9d46-a9fdf52ba5bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:15.892613 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeff1f5-dcb0-4d80-9d46-a9fdf52ba5bd admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:15.920068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1020/2057] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:15 2022] POST /identity/v3/users => generated 301 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:15.924215 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d7bb1-9101-4cda-9f51-1c3c2b4e0d2c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:15.935650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d7bb1-9101-4cda-9f51-1c3c2b4e0d2c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:15.940483 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ce59c-7bff-4cce-ac7e-8f81a934b987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:15.941098 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ce59c-7bff-4cce-ac7e-8f81a934b987 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:15.942146 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ce59c-7bff-4cce-ac7e-8f81a934b987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:15.942469 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ce59c-7bff-4cce-ac7e-8f81a934b987 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:15.943153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ce59c-7bff-4cce-ac7e-8f81a934b987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:15.944749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ce59c-7bff-4cce-ac7e-8f81a934b987 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:15.956670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1038/2058] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:24:15.961171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeff1f5-dcb0-4d80-9d46-a9fdf52ba5bd admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:15.972185 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeff1f5-dcb0-4d80-9d46-a9fdf52ba5bd admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:15.977694 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45633b27-fd2c-4a10-85aa-1408a5f3a540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:15.978688 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45633b27-fd2c-4a10-85aa-1408a5f3a540 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:15.979201 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45633b27-fd2c-4a10-85aa-1408a5f3a540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:15.979673 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45633b27-fd2c-4a10-85aa-1408a5f3a540 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:15.980334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45633b27-fd2c-4a10-85aa-1408a5f3a540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:15.981992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45633b27-fd2c-4a10-85aa-1408a5f3a540 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:15.994952 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1021/2059] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:15.999164 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ce59c-7bff-4cce-ac7e-8f81a934b987 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:16.010946 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ce59c-7bff-4cce-ac7e-8f81a934b987 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:16.016083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c71c22-6924-4d58-9960-992a57e74027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:16.016770 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c71c22-6924-4d58-9960-992a57e74027 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:16.017034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c71c22-6924-4d58-9960-992a57e74027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:16.017360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c71c22-6924-4d58-9960-992a57e74027 admin admin] PATH_INFO: `/v3/projects/0d90e58e3ca54b3b86aa6aa4d4d039fc/users/8c30f5cb5187487e98c688161eda5c6b/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:16.035417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c71c22-6924-4d58-9960-992a57e74027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d90e58e3ca54b3b86aa6aa4d4d039fc, user_id=8c30f5cb5187487e98c688161eda5c6b, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:16.038441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c71c22-6924-4d58-9960-992a57e74027 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:16.057205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1039/2060] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:15 2022] PUT /identity/v3/projects/0d90e58e3ca54b3b86aa6aa4d4d039fc/users/8c30f5cb5187487e98c688161eda5c6b/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:24:16.066156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ec494-5527-427c-a2ab-0bb1ca1c3d89 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:16.067534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ec494-5527-427c-a2ab-0bb1ca1c3d89 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:16.067817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ec494-5527-427c-a2ab-0bb1ca1c3d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:16.160563 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-328ec494-5527-427c-a2ab-0bb1ca1c3d89 None None] MFA Rules not processed for user `8c30f5cb5187487e98c688161eda5c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:16.162120 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-328ec494-5527-427c-a2ab-0bb1ca1c3d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:16.274322 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:16.330433 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1022/2061] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:16 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:16.335486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c71c22-6924-4d58-9960-992a57e74027 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:16.393175 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:16.507448 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c71c22-6924-4d58-9960-992a57e74027 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:16.533098 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ff7f4c-7495-4105-9c54-0f3c4b748ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:16.533676 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff7f4c-7495-4105-9c54-0f3c4b748ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:16.533955 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff7f4c-7495-4105-9c54-0f3c4b748ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:16.534225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ff7f4c-7495-4105-9c54-0f3c4b748ad5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:16.534871 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff7f4c-7495-4105-9c54-0f3c4b748ad5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:16.536855 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ff7f4c-7495-4105-9c54-0f3c4b748ad5 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:16.609209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1040/2062] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:16 2022] POST /identity/v3/projects => generated 397 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:16.614720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ec494-5527-427c-a2ab-0bb1ca1c3d89 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:16.629866 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ec494-5527-427c-a2ab-0bb1ca1c3d89 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:16.724845 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868624f-66da-419c-9dfa-e287f97c28c8 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:16.725233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868624f-66da-419c-9dfa-e287f97c28c8 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:16.725587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868624f-66da-419c-9dfa-e287f97c28c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:16.736003 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a478bb3-80b1-478b-98b6-a80eb85ce8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:16.738182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a478bb3-80b1-478b-98b6-a80eb85ce8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:16.741936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a478bb3-80b1-478b-98b6-a80eb85ce8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:16.742052 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a478bb3-80b1-478b-98b6-a80eb85ce8c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:16.745883 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a478bb3-80b1-478b-98b6-a80eb85ce8c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:16.748614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a478bb3-80b1-478b-98b6-a80eb85ce8c3 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:16.771355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7868624f-66da-419c-9dfa-e287f97c28c8 None None] MFA Rules not processed for user `93fb252cab314529a97fb513e8fa28b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:16.777489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7868624f-66da-419c-9dfa-e287f97c28c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:16.783032 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1023/2063] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:16 2022] POST /identity/v3/users => generated 305 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:16.788655 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a478bb3-80b1-478b-98b6-a80eb85ce8c3 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:16.801539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a478bb3-80b1-478b-98b6-a80eb85ce8c3 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:16.808855 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe485a-9204-4663-8682-3865e57f719b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:16.809996 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe485a-9204-4663-8682-3865e57f719b admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:16.810654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe485a-9204-4663-8682-3865e57f719b admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:16.811117 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe485a-9204-4663-8682-3865e57f719b admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:16.811985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe485a-9204-4663-8682-3865e57f719b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:16.814638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe485a-9204-4663-8682-3865e57f719b admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:16.828993 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1024/2064] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:24:16.833261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe485a-9204-4663-8682-3865e57f719b admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:16.845462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe485a-9204-4663-8682-3865e57f719b admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:16.852267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dab8d7-ab6e-44b2-a9a0-a52d4c2f1589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:16.853226 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dab8d7-ab6e-44b2-a9a0-a52d4c2f1589 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:16.854064 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dab8d7-ab6e-44b2-a9a0-a52d4c2f1589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:16.854855 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dab8d7-ab6e-44b2-a9a0-a52d4c2f1589 admin admin] PATH_INFO: `/v3/projects/403641432a044d268045e561d8188c59/users/e4356c82162e406e9beca4caf626597b/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:16.879302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dab8d7-ab6e-44b2-a9a0-a52d4c2f1589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=403641432a044d268045e561d8188c59, user_id=e4356c82162e406e9beca4caf626597b, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:16.882432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dab8d7-ab6e-44b2-a9a0-a52d4c2f1589 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:16.889343 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1041/2065] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:16.900820 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1025/2066] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:16 2022] PUT /identity/v3/projects/403641432a044d268045e561d8188c59/users/e4356c82162e406e9beca4caf626597b/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:24:16.904295 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868624f-66da-419c-9dfa-e287f97c28c8 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:16.904944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dab8d7-ab6e-44b2-a9a0-a52d4c2f1589 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:16.906171 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7868624f-66da-419c-9dfa-e287f97c28c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:17.005007 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dab8d7-ab6e-44b2-a9a0-a52d4c2f1589 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:17.015673 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15170c25-dae3-425e-acfb-451af8b0b2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:17.016559 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15170c25-dae3-425e-acfb-451af8b0b2cf admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:17.016984 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15170c25-dae3-425e-acfb-451af8b0b2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:17.017456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15170c25-dae3-425e-acfb-451af8b0b2cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:17.018340 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15170c25-dae3-425e-acfb-451af8b0b2cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:17.020391 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868624f-66da-419c-9dfa-e287f97c28c8 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:17.022077 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15170c25-dae3-425e-acfb-451af8b0b2cf admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:17.039144 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1026/2067] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:17.040233 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15170c25-dae3-425e-acfb-451af8b0b2cf admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:17.053549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15170c25-dae3-425e-acfb-451af8b0b2cf admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:17.060127 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0e560-ce83-4d85-9d74-0b76b64f5615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:17.060923 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0e560-ce83-4d85-9d74-0b76b64f5615 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:17.061395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0e560-ce83-4d85-9d74-0b76b64f5615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:17.061730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0e560-ce83-4d85-9d74-0b76b64f5615 admin admin] PATH_INFO: `/v3/projects/403641432a044d268045e561d8188c59/users/e4356c82162e406e9beca4caf626597b/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:17.069687 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:17.069848 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0e560-ce83-4d85-9d74-0b76b64f5615 admin admin] RBAC: Authorizing `identity:create_grant(project_id=403641432a044d268045e561d8188c59, user_id=e4356c82162e406e9beca4caf626597b, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:17.069848 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0e560-ce83-4d85-9d74-0b76b64f5615 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:17.111526 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1027/2068] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:17 2022] PUT /identity/v3/projects/403641432a044d268045e561d8188c59/users/e4356c82162e406e9beca4caf626597b/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:17.115227 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b5a65-2569-49d6-b4a2-3321ee39c7b4 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:17.116684 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b5a65-2569-49d6-b4a2-3321ee39c7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:17.116978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b5a65-2569-49d6-b4a2-3321ee39c7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:17.117288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efdc52-55d8-4a41-aa93-ad8c4e4c73fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:17.118046 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efdc52-55d8-4a41-aa93-ad8c4e4c73fb service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:17.118332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efdc52-55d8-4a41-aa93-ad8c4e4c73fb service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:17.119077 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efdc52-55d8-4a41-aa93-ad8c4e4c73fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:17.126696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a1efdc52-55d8-4a41-aa93-ad8c4e4c73fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:17.175875 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-d39b5a65-2569-49d6-b4a2-3321ee39c7b4 None None] MFA Rules not processed for user `e4356c82162e406e9beca4caf626597b`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:17.177341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d39b5a65-2569-49d6-b4a2-3321ee39c7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:17.182591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:17.233574 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efdc52-55d8-4a41-aa93-ad8c4e4c73fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:17.235254 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efdc52-55d8-4a41-aa93-ad8c4e4c73fb service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:17.247750 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1042/2069] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:17.289846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efdc52-55d8-4a41-aa93-ad8c4e4c73fb service nova] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:17.291592 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a1efdc52-55d8-4a41-aa93-ad8c4e4c73fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:17.329778 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efdc52-55d8-4a41-aa93-ad8c4e4c73fb service nova] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:17.372963 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1028/2070] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:17 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:17.464175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b5a65-2569-49d6-b4a2-3321ee39c7b4 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:17.465848 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d39b5a65-2569-49d6-b4a2-3321ee39c7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:17.496074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a83be4-2e08-4732-bc08-9bd12ff8f637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:17.496074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a83be4-2e08-4732-bc08-9bd12ff8f637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:17.496074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a83be4-2e08-4732-bc08-9bd12ff8f637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:17.496074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a83be4-2e08-4732-bc08-9bd12ff8f637 admin admin] PATH_INFO: `/v3/users/dc63407e186745fea7225f08f9f693c4` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:17.496400 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a83be4-2e08-4732-bc08-9bd12ff8f637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc63407e186745fea7225f08f9f693c4)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:17.496400 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a83be4-2e08-4732-bc08-9bd12ff8f637 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:17.592711 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-45a83be4-2e08-4732-bc08-9bd12ff8f637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc63407e186745fea7225f08f9f693c4'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:17.593508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-45a83be4-2e08-4732-bc08-9bd12ff8f637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc63407e186745fea7225f08f9f693c4'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:17.606555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-45a83be4-2e08-4732-bc08-9bd12ff8f637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc63407e186745fea7225f08f9f693c4'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:17.607441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-45a83be4-2e08-4732-bc08-9bd12ff8f637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc63407e186745fea7225f08f9f693c4'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:17.622097 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-45a83be4-2e08-4732-bc08-9bd12ff8f637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc63407e186745fea7225f08f9f693c4'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:17.632248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:17.664106 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1043/2071] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:17 2022] DELETE /identity/v3/users/dc63407e186745fea7225f08f9f693c4 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:17.670827 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631c413-41be-44ec-86d6-892a432497dc None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:17.673655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631c413-41be-44ec-86d6-892a432497dc None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:17.673938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631c413-41be-44ec-86d6-892a432497dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:17.679076 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b5a65-2569-49d6-b4a2-3321ee39c7b4 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:17.721889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7631c413-41be-44ec-86d6-892a432497dc None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:17.728002 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7631c413-41be-44ec-86d6-892a432497dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:17.752105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda664f-c605-4834-83de-5beca0c293b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:17.756807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda664f-c605-4834-83de-5beca0c293b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:17.757417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda664f-c605-4834-83de-5beca0c293b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:17.757787 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda664f-c605-4834-83de-5beca0c293b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:17.760156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5bda664f-c605-4834-83de-5beca0c293b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:17.870972 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1044/2072] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:17.873889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda664f-c605-4834-83de-5beca0c293b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:17.878124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda664f-c605-4834-83de-5beca0c293b1 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:17.893866 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1029/2073] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:17 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:24:17.909649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631c413-41be-44ec-86d6-892a432497dc None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:17.911918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7631c413-41be-44ec-86d6-892a432497dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:18.061992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631c413-41be-44ec-86d6-892a432497dc None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:18.063760 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda664f-c605-4834-83de-5beca0c293b1 service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:18.065247 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5bda664f-c605-4834-83de-5beca0c293b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:18.073602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda1487-149e-4d63-a6e2-8196381263d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:18.074877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda1487-149e-4d63-a6e2-8196381263d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:18.075314 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda1487-149e-4d63-a6e2-8196381263d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:18.075642 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda1487-149e-4d63-a6e2-8196381263d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:18.083657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda664f-c605-4834-83de-5beca0c293b1 service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:18.093306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5c16f-f87f-4838-afba-4967c991c2ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:18.094693 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5c16f-f87f-4838-afba-4967c991c2ca service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:18.094966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5c16f-f87f-4838-afba-4967c991c2ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:18.095241 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5c16f-f87f-4838-afba-4967c991c2ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:18.099786 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e7c5c16f-f87f-4838-afba-4967c991c2ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:18.112423 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda1487-149e-4d63-a6e2-8196381263d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:18.117248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda1487-149e-4d63-a6e2-8196381263d5 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:18.120486 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:18.147052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1045/2074] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:18.266243 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5c16f-f87f-4838-afba-4967c991c2ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:18.272893 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5c16f-f87f-4838-afba-4967c991c2ca service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:18.298145 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1030/2075] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:18 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:18.743977 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda1487-149e-4d63-a6e2-8196381263d5 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:18.745510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-6dda1487-149e-4d63-a6e2-8196381263d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:18.781832 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda1487-149e-4d63-a6e2-8196381263d5 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:18.802486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd74ed-8f11-44d2-a17b-11c4101be5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:18.803181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd74ed-8f11-44d2-a17b-11c4101be5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:18.805597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd74ed-8f11-44d2-a17b-11c4101be5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:18.805899 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd74ed-8f11-44d2-a17b-11c4101be5e1 admin admin] PATH_INFO: `/v3/projects/a3e85056b9b84206b02ab50803269765` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:18.809232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd74ed-8f11-44d2-a17b-11c4101be5e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3e85056b9b84206b02ab50803269765)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:18.811061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd74ed-8f11-44d2-a17b-11c4101be5e1 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:18.891836 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-fabd74ed-8f11-44d2-a17b-11c4101be5e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3e85056b9b84206b02ab50803269765', 'request_id': 'req-fabd74ed-8f11-44d2-a17b-11c4101be5e1'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:18.902988 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-fabd74ed-8f11-44d2-a17b-11c4101be5e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3e85056b9b84206b02ab50803269765', 'request_id': 'req-fabd74ed-8f11-44d2-a17b-11c4101be5e1'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:18.926972 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-fabd74ed-8f11-44d2-a17b-11c4101be5e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3e85056b9b84206b02ab50803269765', 'request_id': 'req-fabd74ed-8f11-44d2-a17b-11c4101be5e1'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:18.939645 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-fabd74ed-8f11-44d2-a17b-11c4101be5e1 admin admin] The token cache is being invalidate because project a3e85056b9b84206b02ab50803269765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:18.939934 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-fabd74ed-8f11-44d2-a17b-11c4101be5e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:18.940766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-fabd74ed-8f11-44d2-a17b-11c4101be5e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:18.943476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1046/2076] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:18 2022] DELETE /identity/v3/projects/a3e85056b9b84206b02ab50803269765 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:18.948329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641dee6-d6a3-421c-8fb0-4387fff9fec5 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:18.948954 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641dee6-d6a3-421c-8fb0-4387fff9fec5 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:18.949431 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641dee6-d6a3-421c-8fb0-4387fff9fec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:18.975248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-7641dee6-d6a3-421c-8fb0-4387fff9fec5 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:18.976764 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7641dee6-d6a3-421c-8fb0-4387fff9fec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:19.080512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1031/2077] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:19.088917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3c25f-e267-42c0-bcca-7911690aac8e None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:19.089417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3c25f-e267-42c0-bcca-7911690aac8e None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:19.089524 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3c25f-e267-42c0-bcca-7911690aac8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:19.121838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-2de3c25f-e267-42c0-bcca-7911690aac8e None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:19.123909 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2de3c25f-e267-42c0-bcca-7911690aac8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:19.146559 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1047/2078] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:19.150592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641dee6-d6a3-421c-8fb0-4387fff9fec5 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:19.161316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641dee6-d6a3-421c-8fb0-4387fff9fec5 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:19.169613 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbca95e-ba42-407c-a57a-cdddd06f15f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:19.170663 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbca95e-ba42-407c-a57a-cdddd06f15f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:19.171131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbca95e-ba42-407c-a57a-cdddd06f15f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:19.171689 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbca95e-ba42-407c-a57a-cdddd06f15f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:19.172629 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbca95e-ba42-407c-a57a-cdddd06f15f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:24:19.173381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbca95e-ba42-407c-a57a-cdddd06f15f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:19.177465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbca95e-ba42-407c-a57a-cdddd06f15f2 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:19.192869 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1032/2079] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:24:19.207320 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3c25f-e267-42c0-bcca-7911690aac8e None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:19.233857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3c25f-e267-42c0-bcca-7911690aac8e None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:19.250231 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7121ae4-5309-4c5a-bc5e-17558165ebb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:19.250878 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7121ae4-5309-4c5a-bc5e-17558165ebb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:19.251606 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7121ae4-5309-4c5a-bc5e-17558165ebb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:19.251889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7121ae4-5309-4c5a-bc5e-17558165ebb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:19.252534 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7121ae4-5309-4c5a-bc5e-17558165ebb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:19.255368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7121ae4-5309-4c5a-bc5e-17558165ebb1 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:19.262853 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:19.295930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1048/2080] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:19 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:19.300261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbca95e-ba42-407c-a57a-cdddd06f15f2 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:19.320743 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbca95e-ba42-407c-a57a-cdddd06f15f2 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:19.340178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44555ede-7a20-4025-8a64-bf6ee03be66c None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:19.340715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44555ede-7a20-4025-8a64-bf6ee03be66c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:19.340992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44555ede-7a20-4025-8a64-bf6ee03be66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:19.378226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-44555ede-7a20-4025-8a64-bf6ee03be66c None None] MFA Rules not processed for user `49280c781ba745aa83bf4c5db9359973`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:19.382245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-44555ede-7a20-4025-8a64-bf6ee03be66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:19.451138 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11433a68-71fb-429e-aa6a-8ae1dc3ef671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:19.452054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11433a68-71fb-429e-aa6a-8ae1dc3ef671 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:19.452484 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11433a68-71fb-429e-aa6a-8ae1dc3ef671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:19.452900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11433a68-71fb-429e-aa6a-8ae1dc3ef671 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:19.453842 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11433a68-71fb-429e-aa6a-8ae1dc3ef671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:19.456021 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11433a68-71fb-429e-aa6a-8ae1dc3ef671 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:19.486495 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1033/2081] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:19 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:19.489706 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11433a68-71fb-429e-aa6a-8ae1dc3ef671 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:19.506394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1049/2082] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:19.526771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11433a68-71fb-429e-aa6a-8ae1dc3ef671 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:19.549695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e46d24-587a-44b8-9ba2-3fcd6129bec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:19.551493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e46d24-587a-44b8-9ba2-3fcd6129bec9 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:19.551960 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e46d24-587a-44b8-9ba2-3fcd6129bec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:19.552383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e46d24-587a-44b8-9ba2-3fcd6129bec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:19.553218 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e46d24-587a-44b8-9ba2-3fcd6129bec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:19.563848 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e46d24-587a-44b8-9ba2-3fcd6129bec9 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:19.589714 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1034/2083] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:19.591125 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44555ede-7a20-4025-8a64-bf6ee03be66c None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:19.604604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44555ede-7a20-4025-8a64-bf6ee03be66c None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:19.618353 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bed266-8535-4ae9-97af-5bb5bd4da554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:19.618648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bed266-8535-4ae9-97af-5bb5bd4da554 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:19.618924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bed266-8535-4ae9-97af-5bb5bd4da554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:19.619196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bed266-8535-4ae9-97af-5bb5bd4da554 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:19.620604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bed266-8535-4ae9-97af-5bb5bd4da554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:19.636920 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bed266-8535-4ae9-97af-5bb5bd4da554 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:19.643816 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e46d24-587a-44b8-9ba2-3fcd6129bec9 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:19.645910 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-54e46d24-587a-44b8-9ba2-3fcd6129bec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:19.655182 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1050/2084] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:19.656873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bed266-8535-4ae9-97af-5bb5bd4da554 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:19.687755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bed266-8535-4ae9-97af-5bb5bd4da554 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:19.695172 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bab54-4bb2-4964-8286-af9b106483fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:19.696782 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bab54-4bb2-4964-8286-af9b106483fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:19.697254 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bab54-4bb2-4964-8286-af9b106483fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:19.697646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bab54-4bb2-4964-8286-af9b106483fd admin admin] PATH_INFO: `/v3/projects/19f9f5faf55a4b0cb17bf36d8cb838f6/users/2ab18214df00412bb7440535ac725056/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:19.701163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:19.728859 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bab54-4bb2-4964-8286-af9b106483fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f9f5faf55a4b0cb17bf36d8cb838f6, user_id=2ab18214df00412bb7440535ac725056, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:19.731975 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bab54-4bb2-4964-8286-af9b106483fd admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:19.755715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1051/2085] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:19 2022] PUT /identity/v3/projects/19f9f5faf55a4b0cb17bf36d8cb838f6/users/2ab18214df00412bb7440535ac725056/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:24:19.765556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a3700-c5ff-492f-bef4-f8919daf4ca2 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:19.765843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a3700-c5ff-492f-bef4-f8919daf4ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:19.768199 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a3700-c5ff-492f-bef4-f8919daf4ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:19.805590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-8c7a3700-c5ff-492f-bef4-f8919daf4ca2 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:19.807345 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-8c7a3700-c5ff-492f-bef4-f8919daf4ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:19.824582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e46d24-587a-44b8-9ba2-3fcd6129bec9 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:19.926271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9d98e-8097-4ccd-8086-c8bedbb056af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:19.927205 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9d98e-8097-4ccd-8086-c8bedbb056af service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:19.929073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9d98e-8097-4ccd-8086-c8bedbb056af service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:19.929393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9d98e-8097-4ccd-8086-c8bedbb056af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:19.939217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0bc9d98e-8097-4ccd-8086-c8bedbb056af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:19.939631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1052/2086] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:19.951753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a3700-c5ff-492f-bef4-f8919daf4ca2 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:19.957715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-8c7a3700-c5ff-492f-bef4-f8919daf4ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:20.056674 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a3700-c5ff-492f-bef4-f8919daf4ca2 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:20.062408 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f1c73-a580-43a0-981e-d888b2f2ca5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:20.063110 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f1c73-a580-43a0-981e-d888b2f2ca5f service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:20.063329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f1c73-a580-43a0-981e-d888b2f2ca5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:20.063828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f1c73-a580-43a0-981e-d888b2f2ca5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:20.077777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f1c73-a580-43a0-981e-d888b2f2ca5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:20.080179 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f1c73-a580-43a0-981e-d888b2f2ca5f service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:20.087899 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:20.105720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1053/2087] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:20.116787 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9d98e-8097-4ccd-8086-c8bedbb056af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:20.121446 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9d98e-8097-4ccd-8086-c8bedbb056af service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:20.133121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1035/2088] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:21.637980 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f1c73-a580-43a0-981e-d888b2f2ca5f service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:21.640340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-870f1c73-a580-43a0-981e-d888b2f2ca5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:21.674000 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f1c73-a580-43a0-981e-d888b2f2ca5f service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:21.687675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97695f-d359-4269-96f7-7476b2822e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:21.688761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97695f-d359-4269-96f7-7476b2822e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:21.689226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97695f-d359-4269-96f7-7476b2822e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:21.689605 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97695f-d359-4269-96f7-7476b2822e18 admin admin] PATH_INFO: `/v3/users/4cda5c4a66264ee0816618dccb0011ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:21.693669 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97695f-d359-4269-96f7-7476b2822e18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cda5c4a66264ee0816618dccb0011ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:21.695853 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97695f-d359-4269-96f7-7476b2822e18 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:21.858947 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0b97695f-d359-4269-96f7-7476b2822e18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cda5c4a66264ee0816618dccb0011ec'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:21.859771 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0b97695f-d359-4269-96f7-7476b2822e18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cda5c4a66264ee0816618dccb0011ec'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:21.873376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0b97695f-d359-4269-96f7-7476b2822e18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cda5c4a66264ee0816618dccb0011ec'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:21.874638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0b97695f-d359-4269-96f7-7476b2822e18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cda5c4a66264ee0816618dccb0011ec'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:21.888393 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0b97695f-d359-4269-96f7-7476b2822e18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cda5c4a66264ee0816618dccb0011ec'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:21.925530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1054/2089] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:21 2022] DELETE /identity/v3/users/4cda5c4a66264ee0816618dccb0011ec => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:21.932871 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1832356-495f-49f5-8e3c-c8ed7cb9c7e0 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:21.934319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1832356-495f-49f5-8e3c-c8ed7cb9c7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:21.934802 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1832356-495f-49f5-8e3c-c8ed7cb9c7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:21.966699 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e1832356-495f-49f5-8e3c-c8ed7cb9c7e0 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:21.970625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e1832356-495f-49f5-8e3c-c8ed7cb9c7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:22.168426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1036/2090] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:22.177937 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97695f-d359-4269-96f7-7476b2822e18 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:22.185773 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-0b97695f-d359-4269-96f7-7476b2822e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:22.397425 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97695f-d359-4269-96f7-7476b2822e18 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:22.407387 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b99cd-821e-4acb-9827-b276ba4bd439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:22.408017 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b99cd-821e-4acb-9827-b276ba4bd439 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:22.408302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b99cd-821e-4acb-9827-b276ba4bd439 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:22.408572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b99cd-821e-4acb-9827-b276ba4bd439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:22.431724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b99cd-821e-4acb-9827-b276ba4bd439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:22.433778 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b99cd-821e-4acb-9827-b276ba4bd439 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:22.475716 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1055/2091] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:23.296901 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1832356-495f-49f5-8e3c-c8ed7cb9c7e0 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:23.298463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e1832356-495f-49f5-8e3c-c8ed7cb9c7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:23.329357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1832356-495f-49f5-8e3c-c8ed7cb9c7e0 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:23.360685 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951c2a9-4b82-48ff-bd65-a16854e71b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:23.361484 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951c2a9-4b82-48ff-bd65-a16854e71b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:23.361865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951c2a9-4b82-48ff-bd65-a16854e71b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:23.362239 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951c2a9-4b82-48ff-bd65-a16854e71b85 admin admin] PATH_INFO: `/v3/projects/fbfa9c57f947478b9eda820cf3f89303` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:23.365899 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951c2a9-4b82-48ff-bd65-a16854e71b85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbfa9c57f947478b9eda820cf3f89303)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:23.368431 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951c2a9-4b82-48ff-bd65-a16854e71b85 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:23.434558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:23.524100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f951c2a9-4b82-48ff-bd65-a16854e71b85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbfa9c57f947478b9eda820cf3f89303', 'request_id': 'req-f951c2a9-4b82-48ff-bd65-a16854e71b85'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:23.545821 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f951c2a9-4b82-48ff-bd65-a16854e71b85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbfa9c57f947478b9eda820cf3f89303', 'request_id': 'req-f951c2a9-4b82-48ff-bd65-a16854e71b85'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:23.579781 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f951c2a9-4b82-48ff-bd65-a16854e71b85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbfa9c57f947478b9eda820cf3f89303', 'request_id': 'req-f951c2a9-4b82-48ff-bd65-a16854e71b85'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:23.604441 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f951c2a9-4b82-48ff-bd65-a16854e71b85 admin admin] The token cache is being invalidate because project fbfa9c57f947478b9eda820cf3f89303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:23.604790 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f951c2a9-4b82-48ff-bd65-a16854e71b85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:23.606345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f951c2a9-4b82-48ff-bd65-a16854e71b85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:23.610947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1037/2092] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:23 2022] DELETE /identity/v3/projects/fbfa9c57f947478b9eda820cf3f89303 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:23.614721 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b99cd-821e-4acb-9827-b276ba4bd439 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:23.629292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b99cd-821e-4acb-9827-b276ba4bd439 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:23.898163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32672e-2893-4fee-b36a-ae6d92486d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:23.898986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32672e-2893-4fee-b36a-ae6d92486d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:23.899367 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32672e-2893-4fee-b36a-ae6d92486d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:23.899732 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32672e-2893-4fee-b36a-ae6d92486d65 admin admin] PATH_INFO: `/v3/users/64e1d753f61e4b97b72ac163078b6a6f` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:23.902946 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32672e-2893-4fee-b36a-ae6d92486d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64e1d753f61e4b97b72ac163078b6a6f)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:23.905677 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32672e-2893-4fee-b36a-ae6d92486d65 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:23.921371 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951c2a9-4b82-48ff-bd65-a16854e71b85 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:23.923154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f951c2a9-4b82-48ff-bd65-a16854e71b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:23.963912 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951c2a9-4b82-48ff-bd65-a16854e71b85 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:23.986761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:23.989009 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c69e0-38b1-4c41-8ff8-98b464fca1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:23.989797 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c69e0-38b1-4c41-8ff8-98b464fca1fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:23.990057 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c69e0-38b1-4c41-8ff8-98b464fca1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:23.990358 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c69e0-38b1-4c41-8ff8-98b464fca1fb admin admin] PATH_INFO: `/v3/users/f2dd322872da48dc81d9e39948596e53` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:24.002622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c69e0-38b1-4c41-8ff8-98b464fca1fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2dd322872da48dc81d9e39948596e53)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:24.005658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c69e0-38b1-4c41-8ff8-98b464fca1fb admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:24.086709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0d32672e-2893-4fee-b36a-ae6d92486d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64e1d753f61e4b97b72ac163078b6a6f'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:24.090685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0d32672e-2893-4fee-b36a-ae6d92486d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64e1d753f61e4b97b72ac163078b6a6f'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:24.118200 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0d32672e-2893-4fee-b36a-ae6d92486d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64e1d753f61e4b97b72ac163078b6a6f'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:24.118829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0d32672e-2893-4fee-b36a-ae6d92486d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64e1d753f61e4b97b72ac163078b6a6f'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:24.143416 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0d32672e-2893-4fee-b36a-ae6d92486d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64e1d753f61e4b97b72ac163078b6a6f'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:24.186841 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-223c69e0-38b1-4c41-8ff8-98b464fca1fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2dd322872da48dc81d9e39948596e53'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:24.188009 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-223c69e0-38b1-4c41-8ff8-98b464fca1fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2dd322872da48dc81d9e39948596e53'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:24.207918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1056/2093] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:23 2022] DELETE /identity/v3/users/64e1d753f61e4b97b72ac163078b6a6f => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:24.225241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32672e-2893-4fee-b36a-ae6d92486d65 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:24.240340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-0d32672e-2893-4fee-b36a-ae6d92486d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:24.280899 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-223c69e0-38b1-4c41-8ff8-98b464fca1fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2dd322872da48dc81d9e39948596e53'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:24.307921 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-223c69e0-38b1-4c41-8ff8-98b464fca1fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2dd322872da48dc81d9e39948596e53'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:24.310578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-223c69e0-38b1-4c41-8ff8-98b464fca1fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2dd322872da48dc81d9e39948596e53'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:24.361486 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1038/2094] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:23 2022] DELETE /identity/v3/users/f2dd322872da48dc81d9e39948596e53 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:24.371032 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfd512-22c3-4013-a979-cd725708b961 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:24.376685 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfd512-22c3-4013-a979-cd725708b961 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:24.377558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfd512-22c3-4013-a979-cd725708b961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:24.455201 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-ccdfd512-22c3-4013-a979-cd725708b961 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:24.460946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-ccdfd512-22c3-4013-a979-cd725708b961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:24.506127 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32672e-2893-4fee-b36a-ae6d92486d65 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:24.597290 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1039/2095] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:24.649935 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfd512-22c3-4013-a979-cd725708b961 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:24.651406 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-ccdfd512-22c3-4013-a979-cd725708b961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:24.682433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305b32f-4c44-4772-90a8-7ba064f71812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:24.683804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305b32f-4c44-4772-90a8-7ba064f71812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:24.684381 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305b32f-4c44-4772-90a8-7ba064f71812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:24.684850 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a305b32f-4c44-4772-90a8-7ba064f71812 admin admin] PATH_INFO: `/v3/users/8c30f5cb5187487e98c688161eda5c6b` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:24.686442 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305b32f-4c44-4772-90a8-7ba064f71812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c30f5cb5187487e98c688161eda5c6b)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:24.688758 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a305b32f-4c44-4772-90a8-7ba064f71812 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:24.702105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:24.814382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfd512-22c3-4013-a979-cd725708b961 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:24.827532 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7bf0b-0356-44b6-9a61-23b32b38d8e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:24.828454 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7bf0b-0356-44b6-9a61-23b32b38d8e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:24.828925 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7bf0b-0356-44b6-9a61-23b32b38d8e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:24.829428 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7bf0b-0356-44b6-9a61-23b32b38d8e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:24.858963 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7bf0b-0356-44b6-9a61-23b32b38d8e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:24.860523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7bf0b-0356-44b6-9a61-23b32b38d8e0 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:24.862524 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:24.869804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-a305b32f-4c44-4772-90a8-7ba064f71812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c30f5cb5187487e98c688161eda5c6b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:24.871054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-a305b32f-4c44-4772-90a8-7ba064f71812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c30f5cb5187487e98c688161eda5c6b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:24.882211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1040/2096] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:24.897399 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-a305b32f-4c44-4772-90a8-7ba064f71812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c30f5cb5187487e98c688161eda5c6b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:24.898279 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-a305b32f-4c44-4772-90a8-7ba064f71812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c30f5cb5187487e98c688161eda5c6b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:24.913017 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-a305b32f-4c44-4772-90a8-7ba064f71812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c30f5cb5187487e98c688161eda5c6b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:24.970348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1057/2097] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:24 2022] DELETE /identity/v3/users/8c30f5cb5187487e98c688161eda5c6b => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:24.975834 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c53112-8ada-4f1a-93a1-0b7cd05f6169 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:24.976680 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c53112-8ada-4f1a-93a1-0b7cd05f6169 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:24.976680 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c53112-8ada-4f1a-93a1-0b7cd05f6169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:25.000640 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e5c53112-8ada-4f1a-93a1-0b7cd05f6169 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:25.006741 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e5c53112-8ada-4f1a-93a1-0b7cd05f6169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:25.053451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305b32f-4c44-4772-90a8-7ba064f71812 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:25.056302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a305b32f-4c44-4772-90a8-7ba064f71812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:25.125307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1041/2098] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:25.135841 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c53112-8ada-4f1a-93a1-0b7cd05f6169 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:25.140030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e5c53112-8ada-4f1a-93a1-0b7cd05f6169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:25.284725 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a305b32f-4c44-4772-90a8-7ba064f71812 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:25.302491 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35207620-93d6-4bbf-a84c-f81ef941d0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:25.303576 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35207620-93d6-4bbf-a84c-f81ef941d0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:25.304099 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35207620-93d6-4bbf-a84c-f81ef941d0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:25.304481 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35207620-93d6-4bbf-a84c-f81ef941d0e2 admin admin] PATH_INFO: `/v3/projects/cd7220eee3ef426ab997351674bfd6f2` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:25.308099 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35207620-93d6-4bbf-a84c-f81ef941d0e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd7220eee3ef426ab997351674bfd6f2)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:25.312401 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35207620-93d6-4bbf-a84c-f81ef941d0e2 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:25.332795 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c53112-8ada-4f1a-93a1-0b7cd05f6169 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:25.340587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d191ba-6508-4ac5-8300-829b7eef3345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:25.342424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d191ba-6508-4ac5-8300-829b7eef3345 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:25.349167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d191ba-6508-4ac5-8300-829b7eef3345 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:25.349820 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d191ba-6508-4ac5-8300-829b7eef3345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:25.373533 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d191ba-6508-4ac5-8300-829b7eef3345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:25.377220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d191ba-6508-4ac5-8300-829b7eef3345 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:25.408700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-35207620-93d6-4bbf-a84c-f81ef941d0e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd7220eee3ef426ab997351674bfd6f2', 'request_id': 'req-35207620-93d6-4bbf-a84c-f81ef941d0e2'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:25.416708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1042/2099] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:25.418407 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d191ba-6508-4ac5-8300-829b7eef3345 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:25.422079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b6d191ba-6508-4ac5-8300-829b7eef3345 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:25.434097 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-35207620-93d6-4bbf-a84c-f81ef941d0e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd7220eee3ef426ab997351674bfd6f2', 'request_id': 'req-35207620-93d6-4bbf-a84c-f81ef941d0e2'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:25.453318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-35207620-93d6-4bbf-a84c-f81ef941d0e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd7220eee3ef426ab997351674bfd6f2', 'request_id': 'req-35207620-93d6-4bbf-a84c-f81ef941d0e2'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:25.463044 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d191ba-6508-4ac5-8300-829b7eef3345 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:25.482617 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-35207620-93d6-4bbf-a84c-f81ef941d0e2 admin admin] The token cache is being invalidate because project cd7220eee3ef426ab997351674bfd6f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:25.482919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-35207620-93d6-4bbf-a84c-f81ef941d0e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:25.485770 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-35207620-93d6-4bbf-a84c-f81ef941d0e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:25.490573 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1058/2100] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:25 2022] DELETE /identity/v3/projects/cd7220eee3ef426ab997351674bfd6f2 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:25.495243 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613710ea-4601-47b1-a24c-e032d0abcf2c None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:25.496345 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613710ea-4601-47b1-a24c-e032d0abcf2c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:25.496783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613710ea-4601-47b1-a24c-e032d0abcf2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:25.532175 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-613710ea-4601-47b1-a24c-e032d0abcf2c None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:25.533602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-613710ea-4601-47b1-a24c-e032d0abcf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:25.600716 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:25.642317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32ea41-5a56-4937-b47f-dd56398f5f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:25.645289 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32ea41-5a56-4937-b47f-dd56398f5f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:25.645587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32ea41-5a56-4937-b47f-dd56398f5f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:25.645868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32ea41-5a56-4937-b47f-dd56398f5f8e admin admin] PATH_INFO: `/v3/projects/e728aab6b16448df8a492b5138872e40` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:25.647272 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32ea41-5a56-4937-b47f-dd56398f5f8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e728aab6b16448df8a492b5138872e40)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:25.650458 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32ea41-5a56-4937-b47f-dd56398f5f8e admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:25.661102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:25.663793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1059/2101] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:25.669208 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ea85d-5d8f-421f-9759-a48fcfa487df None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:25.669441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ea85d-5d8f-421f-9759-a48fcfa487df None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:25.669618 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ea85d-5d8f-421f-9759-a48fcfa487df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:25.703983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-061ea85d-5d8f-421f-9759-a48fcfa487df None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:25.705340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-061ea85d-5d8f-421f-9759-a48fcfa487df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:25.752461 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-fb32ea41-5a56-4937-b47f-dd56398f5f8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e728aab6b16448df8a492b5138872e40', 'request_id': 'req-fb32ea41-5a56-4937-b47f-dd56398f5f8e'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:25.770870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-fb32ea41-5a56-4937-b47f-dd56398f5f8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e728aab6b16448df8a492b5138872e40', 'request_id': 'req-fb32ea41-5a56-4937-b47f-dd56398f5f8e'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:25.789164 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-fb32ea41-5a56-4937-b47f-dd56398f5f8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e728aab6b16448df8a492b5138872e40', 'request_id': 'req-fb32ea41-5a56-4937-b47f-dd56398f5f8e'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:25.813176 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-fb32ea41-5a56-4937-b47f-dd56398f5f8e admin admin] The token cache is being invalidate because project e728aab6b16448df8a492b5138872e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:25.813176 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-fb32ea41-5a56-4937-b47f-dd56398f5f8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:25.817270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-fb32ea41-5a56-4937-b47f-dd56398f5f8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:25.822999 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1043/2102] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:25 2022] DELETE /identity/v3/projects/e728aab6b16448df8a492b5138872e40 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:25.824152 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32ea41-5a56-4937-b47f-dd56398f5f8e admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:25.826965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-fb32ea41-5a56-4937-b47f-dd56398f5f8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:25.899656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1060/2103] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:25.900626 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ea85d-5d8f-421f-9759-a48fcfa487df None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:25.935159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ea85d-5d8f-421f-9759-a48fcfa487df None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:25.955267 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1bf85-5654-4a53-8ce0-692f8837b70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:25.955904 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1bf85-5654-4a53-8ce0-692f8837b70b admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:25.956264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1bf85-5654-4a53-8ce0-692f8837b70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:25.956438 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1bf85-5654-4a53-8ce0-692f8837b70b admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:25.957966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1bf85-5654-4a53-8ce0-692f8837b70b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:24:25.958155 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1bf85-5654-4a53-8ce0-692f8837b70b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:25.962963 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b1bf85-5654-4a53-8ce0-692f8837b70b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:26.001298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1061/2104] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:24:26.002406 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1bf85-5654-4a53-8ce0-692f8837b70b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:26.020991 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-25b1bf85-5654-4a53-8ce0-692f8837b70b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:26.058913 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1bf85-5654-4a53-8ce0-692f8837b70b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:26.062845 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32ea41-5a56-4937-b47f-dd56398f5f8e admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:26.074628 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c70775-f2cd-4e44-8966-f15a7d04b6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:26.075207 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c70775-f2cd-4e44-8966-f15a7d04b6c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:26.075485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c70775-f2cd-4e44-8966-f15a7d04b6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:26.075756 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c70775-f2cd-4e44-8966-f15a7d04b6c6 admin admin] PATH_INFO: `/v3/projects/0d90e58e3ca54b3b86aa6aa4d4d039fc` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:26.077334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c70775-f2cd-4e44-8966-f15a7d04b6c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d90e58e3ca54b3b86aa6aa4d4d039fc)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:26.079270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c70775-f2cd-4e44-8966-f15a7d04b6c6 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:26.088235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8896c-9a1d-4838-bfe3-f0fdad2e439e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:26.088904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8896c-9a1d-4838-bfe3-f0fdad2e439e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:26.089324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8896c-9a1d-4838-bfe3-f0fdad2e439e admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:26.089516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8896c-9a1d-4838-bfe3-f0fdad2e439e admin admin] PATH_INFO: `/v3/users/82d287ba18b146809f4727597e93a1bb` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:26.090889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e8896c-9a1d-4838-bfe3-f0fdad2e439e admin admin] RBAC: Authorizing `identity:delete_user(user_id=82d287ba18b146809f4727597e93a1bb)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:26.092822 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e8896c-9a1d-4838-bfe3-f0fdad2e439e admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:26.228321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-78e8896c-9a1d-4838-bfe3-f0fdad2e439e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82d287ba18b146809f4727597e93a1bb'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:26.229323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-78e8896c-9a1d-4838-bfe3-f0fdad2e439e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82d287ba18b146809f4727597e93a1bb'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:26.231849 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-60c70775-f2cd-4e44-8966-f15a7d04b6c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d90e58e3ca54b3b86aa6aa4d4d039fc', 'request_id': 'req-60c70775-f2cd-4e44-8966-f15a7d04b6c6'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:26.251881 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-78e8896c-9a1d-4838-bfe3-f0fdad2e439e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82d287ba18b146809f4727597e93a1bb'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:26.254685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-60c70775-f2cd-4e44-8966-f15a7d04b6c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d90e58e3ca54b3b86aa6aa4d4d039fc', 'request_id': 'req-60c70775-f2cd-4e44-8966-f15a7d04b6c6'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:26.269787 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-78e8896c-9a1d-4838-bfe3-f0fdad2e439e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82d287ba18b146809f4727597e93a1bb'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:26.271409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-78e8896c-9a1d-4838-bfe3-f0fdad2e439e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82d287ba18b146809f4727597e93a1bb'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:26.273243 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-60c70775-f2cd-4e44-8966-f15a7d04b6c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d90e58e3ca54b3b86aa6aa4d4d039fc', 'request_id': 'req-60c70775-f2cd-4e44-8966-f15a7d04b6c6'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:26.297743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-60c70775-f2cd-4e44-8966-f15a7d04b6c6 admin admin] The token cache is being invalidate because project 0d90e58e3ca54b3b86aa6aa4d4d039fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:26.298099 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-60c70775-f2cd-4e44-8966-f15a7d04b6c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:26.299823 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1044/2105] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:25 2022] DELETE /identity/v3/users/82d287ba18b146809f4727597e93a1bb => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:24:26.300698 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8896c-9a1d-4838-bfe3-f0fdad2e439e admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:26.301493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-60c70775-f2cd-4e44-8966-f15a7d04b6c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:26.302672 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-78e8896c-9a1d-4838-bfe3-f0fdad2e439e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:26.304146 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1062/2106] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:26 2022] DELETE /identity/v3/projects/0d90e58e3ca54b3b86aa6aa4d4d039fc => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:26.310017 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c70775-f2cd-4e44-8966-f15a7d04b6c6 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:26.312502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-60c70775-f2cd-4e44-8966-f15a7d04b6c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:26.319855 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:26.422898 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c70775-f2cd-4e44-8966-f15a7d04b6c6 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:26.431812 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8896c-9a1d-4838-bfe3-f0fdad2e439e admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:26.435793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32208c-8e66-48b2-ad96-418ed740551b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:26.437062 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32208c-8e66-48b2-ad96-418ed740551b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:26.437953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32208c-8e66-48b2-ad96-418ed740551b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:26.437953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32208c-8e66-48b2-ad96-418ed740551b admin admin] PATH_INFO: `/v3/users/e4356c82162e406e9beca4caf626597b` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:26.439394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f32208c-8e66-48b2-ad96-418ed740551b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4356c82162e406e9beca4caf626597b)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:26.443199 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f32208c-8e66-48b2-ad96-418ed740551b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:26.443658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa6d82-8087-421d-a606-93cf074f64af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:26.444294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa6d82-8087-421d-a606-93cf074f64af admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:26.444751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa6d82-8087-421d-a606-93cf074f64af admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:26.445348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa6d82-8087-421d-a606-93cf074f64af admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:26.446141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa6d82-8087-421d-a606-93cf074f64af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:26.448056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:26.450534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa6d82-8087-421d-a606-93cf074f64af admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:26.515034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1045/2107] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:26 2022] POST /identity/v3/projects => generated 379 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:26.520973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa6d82-8087-421d-a606-93cf074f64af admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:26.524655 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8daa6d82-8087-421d-a606-93cf074f64af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:26.549209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8f32208c-8e66-48b2-ad96-418ed740551b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4356c82162e406e9beca4caf626597b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:26.550562 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8f32208c-8e66-48b2-ad96-418ed740551b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4356c82162e406e9beca4caf626597b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:26.576395 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8f32208c-8e66-48b2-ad96-418ed740551b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4356c82162e406e9beca4caf626597b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:26.577339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8f32208c-8e66-48b2-ad96-418ed740551b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4356c82162e406e9beca4caf626597b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:26.591058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8f32208c-8e66-48b2-ad96-418ed740551b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4356c82162e406e9beca4caf626597b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:26.621633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1063/2108] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:26 2022] DELETE /identity/v3/users/e4356c82162e406e9beca4caf626597b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:26.713581 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa6d82-8087-421d-a606-93cf074f64af admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:26.823729 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70f2a0-3317-408c-aade-942a8818be3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:26.824272 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70f2a0-3317-408c-aade-942a8818be3c admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:26.824547 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70f2a0-3317-408c-aade-942a8818be3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:26.824820 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70f2a0-3317-408c-aade-942a8818be3c admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:26.825492 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70f2a0-3317-408c-aade-942a8818be3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:26.827595 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70f2a0-3317-408c-aade-942a8818be3c admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:26.855278 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1046/2109] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:26 2022] POST /identity/v3/users => generated 290 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:26.859734 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32208c-8e66-48b2-ad96-418ed740551b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:26.861846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-8f32208c-8e66-48b2-ad96-418ed740551b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:26.879535 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32208c-8e66-48b2-ad96-418ed740551b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:26.885063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea233787-4946-4243-8f0b-8069d0f609e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:26.885952 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea233787-4946-4243-8f0b-8069d0f609e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:26.885952 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea233787-4946-4243-8f0b-8069d0f609e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:26.886242 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea233787-4946-4243-8f0b-8069d0f609e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:26.886887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea233787-4946-4243-8f0b-8069d0f609e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:26.888487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea233787-4946-4243-8f0b-8069d0f609e2 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:26.899620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1064/2110] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:26.903698 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70f2a0-3317-408c-aade-942a8818be3c admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:26.917919 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70f2a0-3317-408c-aade-942a8818be3c admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:26.932025 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5cfdc-da0b-41bf-aea5-2823ed9abe36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:26.932664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5cfdc-da0b-41bf-aea5-2823ed9abe36 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:26.932947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5cfdc-da0b-41bf-aea5-2823ed9abe36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:26.933253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5cfdc-da0b-41bf-aea5-2823ed9abe36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:26.937649 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5cfdc-da0b-41bf-aea5-2823ed9abe36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:26.939393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5cfdc-da0b-41bf-aea5-2823ed9abe36 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:26.969625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1047/2111] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:24:26.974434 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea233787-4946-4243-8f0b-8069d0f609e2 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:26.991015 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea233787-4946-4243-8f0b-8069d0f609e2 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:27.002220 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead0153-ba01-4ef1-8df7-ba8f6acc7bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:27.002897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead0153-ba01-4ef1-8df7-ba8f6acc7bd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:27.003182 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead0153-ba01-4ef1-8df7-ba8f6acc7bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:27.004069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead0153-ba01-4ef1-8df7-ba8f6acc7bd6 admin admin] PATH_INFO: `/v3/projects/09481cbd5faa43a483e47d61c6b038c9/users/be8b051080db459d9818738fedb4a913/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:27.040539 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead0153-ba01-4ef1-8df7-ba8f6acc7bd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09481cbd5faa43a483e47d61c6b038c9, user_id=be8b051080db459d9818738fedb4a913, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:27.040702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5cfdc-da0b-41bf-aea5-2823ed9abe36 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:27.042735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c4e5cfdc-da0b-41bf-aea5-2823ed9abe36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:27.044011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead0153-ba01-4ef1-8df7-ba8f6acc7bd6 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:27.059202 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5cfdc-da0b-41bf-aea5-2823ed9abe36 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:27.069317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45cade-4a8d-48be-ab9a-a82b8573046f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:27.070163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45cade-4a8d-48be-ab9a-a82b8573046f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:27.070449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45cade-4a8d-48be-ab9a-a82b8573046f admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:27.070783 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45cade-4a8d-48be-ab9a-a82b8573046f admin admin] PATH_INFO: `/v3/projects/ce3a5d2f00c4404784a72a106dd0e17f` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:27.072037 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45cade-4a8d-48be-ab9a-a82b8573046f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce3a5d2f00c4404784a72a106dd0e17f)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:27.074451 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f45cade-4a8d-48be-ab9a-a82b8573046f admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:27.094277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1065/2112] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:26 2022] PUT /identity/v3/projects/09481cbd5faa43a483e47d61c6b038c9/users/be8b051080db459d9818738fedb4a913/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:27.106051 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:27.109332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378826e4-820d-45de-99fa-e88f4aaf9666 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:27.112331 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378826e4-820d-45de-99fa-e88f4aaf9666 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:27.115680 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378826e4-820d-45de-99fa-e88f4aaf9666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:27.161649 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1f45cade-4a8d-48be-ab9a-a82b8573046f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce3a5d2f00c4404784a72a106dd0e17f', 'request_id': 'req-1f45cade-4a8d-48be-ab9a-a82b8573046f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:27.168254 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-378826e4-820d-45de-99fa-e88f4aaf9666 None None] MFA Rules not processed for user `be8b051080db459d9818738fedb4a913`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:27.169529 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-378826e4-820d-45de-99fa-e88f4aaf9666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:27.179667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1f45cade-4a8d-48be-ab9a-a82b8573046f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce3a5d2f00c4404784a72a106dd0e17f', 'request_id': 'req-1f45cade-4a8d-48be-ab9a-a82b8573046f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:27.195410 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1f45cade-4a8d-48be-ab9a-a82b8573046f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce3a5d2f00c4404784a72a106dd0e17f', 'request_id': 'req-1f45cade-4a8d-48be-ab9a-a82b8573046f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:27.221347 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1f45cade-4a8d-48be-ab9a-a82b8573046f admin admin] The token cache is being invalidate because project ce3a5d2f00c4404784a72a106dd0e17f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:27.222122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1f45cade-4a8d-48be-ab9a-a82b8573046f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:27.224745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1f45cade-4a8d-48be-ab9a-a82b8573046f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:27.233823 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1048/2113] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:27 2022] DELETE /identity/v3/projects/ce3a5d2f00c4404784a72a106dd0e17f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:27.237945 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45cade-4a8d-48be-ab9a-a82b8573046f admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:27.240747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1f45cade-4a8d-48be-ab9a-a82b8573046f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:27.251279 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1066/2114] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:27.255877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378826e4-820d-45de-99fa-e88f4aaf9666 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:27.257314 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-378826e4-820d-45de-99fa-e88f4aaf9666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:27.324080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:27.370809 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f45cade-4a8d-48be-ab9a-a82b8573046f admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:27.376744 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23750098-f805-44ed-9d2d-ec72545c9477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:27.377445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23750098-f805-44ed-9d2d-ec72545c9477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:27.377724 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23750098-f805-44ed-9d2d-ec72545c9477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:27.378000 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23750098-f805-44ed-9d2d-ec72545c9477 admin admin] PATH_INFO: `/v3/users/c388ff177cc7407cbdd6b4cfad510b1f` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:27.379940 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23750098-f805-44ed-9d2d-ec72545c9477 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c388ff177cc7407cbdd6b4cfad510b1f)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:27.382484 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378826e4-820d-45de-99fa-e88f4aaf9666 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:27.382660 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23750098-f805-44ed-9d2d-ec72545c9477 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:27.393794 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c4cb6-6b47-44e5-9f36-adaad752ba7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:27.395471 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c4cb6-6b47-44e5-9f36-adaad752ba7a admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:27.396015 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c4cb6-6b47-44e5-9f36-adaad752ba7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:27.396377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c4cb6-6b47-44e5-9f36-adaad752ba7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:27.402260 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c4cb6-6b47-44e5-9f36-adaad752ba7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:27.416083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c4cb6-6b47-44e5-9f36-adaad752ba7a admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:27.454195 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-23750098-f805-44ed-9d2d-ec72545c9477 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c388ff177cc7407cbdd6b4cfad510b1f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:27.466450 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-23750098-f805-44ed-9d2d-ec72545c9477 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c388ff177cc7407cbdd6b4cfad510b1f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:27.488983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-23750098-f805-44ed-9d2d-ec72545c9477 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c388ff177cc7407cbdd6b4cfad510b1f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:27.501041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-23750098-f805-44ed-9d2d-ec72545c9477 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c388ff177cc7407cbdd6b4cfad510b1f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:27.501574 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-23750098-f805-44ed-9d2d-ec72545c9477 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c388ff177cc7407cbdd6b4cfad510b1f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:27.506632 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1067/2115] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:27 2022] POST /identity/v3/projects => generated 379 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:27.509936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c4cb6-6b47-44e5-9f36-adaad752ba7a admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:27.515823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-677c4cb6-6b47-44e5-9f36-adaad752ba7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:27.521038 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1049/2116] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:27 2022] DELETE /identity/v3/users/c388ff177cc7407cbdd6b4cfad510b1f => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:24:27.522031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23750098-f805-44ed-9d2d-ec72545c9477 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:27.523733 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-23750098-f805-44ed-9d2d-ec72545c9477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:27.563181 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:27.625666 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23750098-f805-44ed-9d2d-ec72545c9477 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:27.644277 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de731af1-0529-4cbe-b7de-d71e7b4d809b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:27.645061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de731af1-0529-4cbe-b7de-d71e7b4d809b admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:27.645371 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de731af1-0529-4cbe-b7de-d71e7b4d809b admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:27.645645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de731af1-0529-4cbe-b7de-d71e7b4d809b admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:27.645806 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c4cb6-6b47-44e5-9f36-adaad752ba7a admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:27.646289 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de731af1-0529-4cbe-b7de-d71e7b4d809b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:27.648372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de731af1-0529-4cbe-b7de-d71e7b4d809b admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:27.653253 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f86b2d8-7cf9-43a3-9a4d-8948a29ac6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:27.656204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86b2d8-7cf9-43a3-9a4d-8948a29ac6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:27.656204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86b2d8-7cf9-43a3-9a4d-8948a29ac6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:27.656204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f86b2d8-7cf9-43a3-9a4d-8948a29ac6fd admin admin] PATH_INFO: `/v3/projects/403641432a044d268045e561d8188c59` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:27.656826 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f86b2d8-7cf9-43a3-9a4d-8948a29ac6fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=403641432a044d268045e561d8188c59)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:27.659124 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f86b2d8-7cf9-43a3-9a4d-8948a29ac6fd admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:27.696669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1050/2117] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:27 2022] POST /identity/v3/users => generated 296 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:27.701878 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de731af1-0529-4cbe-b7de-d71e7b4d809b admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:27.717513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de731af1-0529-4cbe-b7de-d71e7b4d809b admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:27.724755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a8287-d637-4f50-aebd-69a4c14b3db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:27.725545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a8287-d637-4f50-aebd-69a4c14b3db4 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:27.725924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a8287-d637-4f50-aebd-69a4c14b3db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:27.726243 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a8287-d637-4f50-aebd-69a4c14b3db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:27.727081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3a8287-d637-4f50-aebd-69a4c14b3db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:27.729824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3a8287-d637-4f50-aebd-69a4c14b3db4 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:27.749474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5f86b2d8-7cf9-43a3-9a4d-8948a29ac6fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '403641432a044d268045e561d8188c59', 'request_id': 'req-5f86b2d8-7cf9-43a3-9a4d-8948a29ac6fd'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:27.757634 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1051/2118] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:24:27.758158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5f86b2d8-7cf9-43a3-9a4d-8948a29ac6fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '403641432a044d268045e561d8188c59', 'request_id': 'req-5f86b2d8-7cf9-43a3-9a4d-8948a29ac6fd'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:27.765219 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a8287-d637-4f50-aebd-69a4c14b3db4 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:27.773231 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5f86b2d8-7cf9-43a3-9a4d-8948a29ac6fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '403641432a044d268045e561d8188c59', 'request_id': 'req-5f86b2d8-7cf9-43a3-9a4d-8948a29ac6fd'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:27.787383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a8287-d637-4f50-aebd-69a4c14b3db4 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:27.793172 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5f86b2d8-7cf9-43a3-9a4d-8948a29ac6fd admin admin] The token cache is being invalidate because project 403641432a044d268045e561d8188c59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:27.793172 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5f86b2d8-7cf9-43a3-9a4d-8948a29ac6fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:27.794944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5f86b2d8-7cf9-43a3-9a4d-8948a29ac6fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:27.797099 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1068/2119] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:27 2022] DELETE /identity/v3/projects/403641432a044d268045e561d8188c59 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:27.802690 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33445e0d-ee8b-48ba-aa43-4d35d71f35cf None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:27.803060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33445e0d-ee8b-48ba-aa43-4d35d71f35cf None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:27.803400 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33445e0d-ee8b-48ba-aa43-4d35d71f35cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:27.834188 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-33445e0d-ee8b-48ba-aa43-4d35d71f35cf None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:27.836476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-33445e0d-ee8b-48ba-aa43-4d35d71f35cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:27.913123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:27.930181 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667260e-daf7-4f3d-ab6c-451735b6d5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:27.931081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667260e-daf7-4f3d-ab6c-451735b6d5dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:27.931363 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667260e-daf7-4f3d-ab6c-451735b6d5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:27.931637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6667260e-daf7-4f3d-ab6c-451735b6d5dd admin admin] PATH_INFO: `/v3/projects/b55a3d1563ff412a8dfe2ef9a9f8f522/users/55335d5c3c424cae94310d439e87e983/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:27.937373 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1069/2120] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:27.945000 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8e829-7bed-4e5a-a598-eda81fba6130 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:27.945309 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8e829-7bed-4e5a-a598-eda81fba6130 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:27.949877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8e829-7bed-4e5a-a598-eda81fba6130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:27.975648 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667260e-daf7-4f3d-ab6c-451735b6d5dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b55a3d1563ff412a8dfe2ef9a9f8f522, user_id=55335d5c3c424cae94310d439e87e983, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:27.978159 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6667260e-daf7-4f3d-ab6c-451735b6d5dd admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:27.993515 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7cf8e829-7bed-4e5a-a598-eda81fba6130 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:27.994737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7cf8e829-7bed-4e5a-a598-eda81fba6130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:27.999987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1052/2121] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:27 2022] PUT /identity/v3/projects/b55a3d1563ff412a8dfe2ef9a9f8f522/users/55335d5c3c424cae94310d439e87e983/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:28.004406 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667260e-daf7-4f3d-ab6c-451735b6d5dd admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:28.005790 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-6667260e-daf7-4f3d-ab6c-451735b6d5dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:28.142281 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1070/2122] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:28.147067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8e829-7bed-4e5a-a598-eda81fba6130 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:28.165191 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6667260e-daf7-4f3d-ab6c-451735b6d5dd admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:28.191543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8e829-7bed-4e5a-a598-eda81fba6130 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:28.201455 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b262a-98d6-4b35-82db-2d4195b58e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:28.202078 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b262a-98d6-4b35-82db-2d4195b58e83 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:28.202360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b262a-98d6-4b35-82db-2d4195b58e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:28.202624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b262a-98d6-4b35-82db-2d4195b58e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:28.203267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b262a-98d6-4b35-82db-2d4195b58e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:28.204873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1b262a-98d6-4b35-82db-2d4195b58e83 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:28.207607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792bc5a-c7e8-40e2-9100-5e8b11bfa081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:28.208697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792bc5a-c7e8-40e2-9100-5e8b11bfa081 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:28.208976 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792bc5a-c7e8-40e2-9100-5e8b11bfa081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:28.209274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792bc5a-c7e8-40e2-9100-5e8b11bfa081 admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:28.209928 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792bc5a-c7e8-40e2-9100-5e8b11bfa081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:24:28.217360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792bc5a-c7e8-40e2-9100-5e8b11bfa081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:28.219380 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1792bc5a-c7e8-40e2-9100-5e8b11bfa081 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:28.253657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1053/2123] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:24:28.255840 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b262a-98d6-4b35-82db-2d4195b58e83 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:28.269719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1071/2124] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:24:28.274651 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792bc5a-c7e8-40e2-9100-5e8b11bfa081 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:28.289988 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b262a-98d6-4b35-82db-2d4195b58e83 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:28.294729 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792bc5a-c7e8-40e2-9100-5e8b11bfa081 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:28.303896 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583ba0d-011f-48c6-b2a9-794d7da6a254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:28.303896 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583ba0d-011f-48c6-b2a9-794d7da6a254 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:28.303896 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583ba0d-011f-48c6-b2a9-794d7da6a254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:28.304318 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583ba0d-011f-48c6-b2a9-794d7da6a254 admin admin] PATH_INFO: `/v3/projects/b55a3d1563ff412a8dfe2ef9a9f8f522/users/55335d5c3c424cae94310d439e87e983/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:28.308204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583ba0d-011f-48c6-b2a9-794d7da6a254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b55a3d1563ff412a8dfe2ef9a9f8f522, user_id=55335d5c3c424cae94310d439e87e983, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:28.324060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede1d87-0435-4a15-bead-8d30adeea85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:28.324060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede1d87-0435-4a15-bead-8d30adeea85f admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:28.324060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede1d87-0435-4a15-bead-8d30adeea85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:28.324060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede1d87-0435-4a15-bead-8d30adeea85f admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:28.324555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede1d87-0435-4a15-bead-8d30adeea85f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:28.324877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583ba0d-011f-48c6-b2a9-794d7da6a254 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:28.328245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede1d87-0435-4a15-bead-8d30adeea85f admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:28.364189 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1054/2125] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:28 2022] PUT /identity/v3/projects/b55a3d1563ff412a8dfe2ef9a9f8f522/users/55335d5c3c424cae94310d439e87e983/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:28.370046 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1072/2126] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:28 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:28.377431 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52e837-e90a-43d6-805c-c77ed974994e None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:28.377720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52e837-e90a-43d6-805c-c77ed974994e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:28.378037 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52e837-e90a-43d6-805c-c77ed974994e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:28.383244 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede1d87-0435-4a15-bead-8d30adeea85f admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:28.430476 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:28.451189 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-aa52e837-e90a-43d6-805c-c77ed974994e None None] MFA Rules not processed for user `55335d5c3c424cae94310d439e87e983`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:28.457936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-aa52e837-e90a-43d6-805c-c77ed974994e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:28.510114 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede1d87-0435-4a15-bead-8d30adeea85f admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:28.523172 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4d0d3-f036-437e-9774-42b15255cf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:28.523803 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4d0d3-f036-437e-9774-42b15255cf9b admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:28.524085 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4d0d3-f036-437e-9774-42b15255cf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:28.524363 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4d0d3-f036-437e-9774-42b15255cf9b admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:28.525061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4d0d3-f036-437e-9774-42b15255cf9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:28.527029 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4d0d3-f036-437e-9774-42b15255cf9b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:28.562876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1073/2127] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:28 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:28.563957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4d0d3-f036-437e-9774-42b15255cf9b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:28.567463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-81c4d0d3-f036-437e-9774-42b15255cf9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:28.588562 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4d0d3-f036-437e-9774-42b15255cf9b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:28.596280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139437b-52ca-417e-af96-55baa058184e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:28.597029 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139437b-52ca-417e-af96-55baa058184e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:28.597356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139437b-52ca-417e-af96-55baa058184e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:28.597635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139437b-52ca-417e-af96-55baa058184e admin admin] PATH_INFO: `/v3/projects/8920549484404e778283415dd9fef34f` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:28.599077 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6139437b-52ca-417e-af96-55baa058184e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8920549484404e778283415dd9fef34f)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:28.602795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6139437b-52ca-417e-af96-55baa058184e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:28.643218 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1055/2128] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:28.645499 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52e837-e90a-43d6-805c-c77ed974994e None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:28.662029 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52e837-e90a-43d6-805c-c77ed974994e None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:28.667673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:28.684759 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6139437b-52ca-417e-af96-55baa058184e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8920549484404e778283415dd9fef34f', 'request_id': 'req-6139437b-52ca-417e-af96-55baa058184e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:28.699306 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6139437b-52ca-417e-af96-55baa058184e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8920549484404e778283415dd9fef34f', 'request_id': 'req-6139437b-52ca-417e-af96-55baa058184e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:28.723978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6139437b-52ca-417e-af96-55baa058184e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8920549484404e778283415dd9fef34f', 'request_id': 'req-6139437b-52ca-417e-af96-55baa058184e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:28.746277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6139437b-52ca-417e-af96-55baa058184e admin admin] The token cache is being invalidate because project 8920549484404e778283415dd9fef34f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:28.746579 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6139437b-52ca-417e-af96-55baa058184e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:28.747363 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6139437b-52ca-417e-af96-55baa058184e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:28.749570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1074/2129] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:28 2022] DELETE /identity/v3/projects/8920549484404e778283415dd9fef34f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:28.750482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139437b-52ca-417e-af96-55baa058184e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:28.752677 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-6139437b-52ca-417e-af96-55baa058184e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:28.760442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccd068-696f-4259-a121-8f2bcf81f7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:28.766151 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccd068-696f-4259-a121-8f2bcf81f7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:28.766833 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccd068-696f-4259-a121-8f2bcf81f7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:28.767924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccd068-696f-4259-a121-8f2bcf81f7c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:28.770262 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccd068-696f-4259-a121-8f2bcf81f7c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:28.772719 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccd068-696f-4259-a121-8f2bcf81f7c7 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:28.803170 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1056/2130] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:28.807219 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43040b1-ddd8-4c3d-a714-979161f701ef None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:28.809168 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43040b1-ddd8-4c3d-a714-979161f701ef None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:28.809632 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43040b1-ddd8-4c3d-a714-979161f701ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:28.869348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-f43040b1-ddd8-4c3d-a714-979161f701ef None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:28.871010 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f43040b1-ddd8-4c3d-a714-979161f701ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:28.884520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1057/2131] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:28.885857 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43040b1-ddd8-4c3d-a714-979161f701ef None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:28.888228 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f43040b1-ddd8-4c3d-a714-979161f701ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:28.895168 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139437b-52ca-417e-af96-55baa058184e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:28.901807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1144bd6-f8da-4ed6-91a6-3d7059e847ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:28.902449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1144bd6-f8da-4ed6-91a6-3d7059e847ac service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:28.902733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1144bd6-f8da-4ed6-91a6-3d7059e847ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:28.903008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1144bd6-f8da-4ed6-91a6-3d7059e847ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:28.906210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e1144bd6-f8da-4ed6-91a6-3d7059e847ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:28.913327 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43040b1-ddd8-4c3d-a714-979161f701ef None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:28.919375 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645d387-a477-4ee8-8158-3378c38c2a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:28.920007 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645d387-a477-4ee8-8158-3378c38c2a87 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:28.920283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645d387-a477-4ee8-8158-3378c38c2a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:28.920552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645d387-a477-4ee8-8158-3378c38c2a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:28.921290 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645d387-a477-4ee8-8158-3378c38c2a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:28.922895 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645d387-a477-4ee8-8158-3378c38c2a87 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:28.953570 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1058/2132] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:28.960015 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8631c77-75aa-4168-a4a0-3a3bc95e2293 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:28.960298 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8631c77-75aa-4168-a4a0-3a3bc95e2293 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:28.960637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8631c77-75aa-4168-a4a0-3a3bc95e2293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:29.015266 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e8631c77-75aa-4168-a4a0-3a3bc95e2293 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:29.016448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e8631c77-75aa-4168-a4a0-3a3bc95e2293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:29.038062 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1059/2133] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:29.038910 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8631c77-75aa-4168-a4a0-3a3bc95e2293 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:29.058310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8631c77-75aa-4168-a4a0-3a3bc95e2293 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:29.062648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1144bd6-f8da-4ed6-91a6-3d7059e847ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:29.064474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1144bd6-f8da-4ed6-91a6-3d7059e847ac service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:29.071489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4109e-37af-4d8f-9476-96322abcdce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:29.072341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4109e-37af-4d8f-9476-96322abcdce5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:29.072623 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4109e-37af-4d8f-9476-96322abcdce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:29.072898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4109e-37af-4d8f-9476-96322abcdce5 admin admin] PATH_INFO: `/v3/projects/a5fc5cc3087f4ba68bba4daa437b7cc6/users/564bc620e923434b9389923314499d6a/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:29.083073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1075/2134] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:29.083651 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1144bd6-f8da-4ed6-91a6-3d7059e847ac service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:29.097441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1144bd6-f8da-4ed6-91a6-3d7059e847ac service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:29.108317 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a1a74-17f2-4346-a332-cca626ba55ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:29.108973 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a1a74-17f2-4346-a332-cca626ba55ac admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:29.109841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a1a74-17f2-4346-a332-cca626ba55ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:29.109966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a1a74-17f2-4346-a332-cca626ba55ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:29.114722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a1a74-17f2-4346-a332-cca626ba55ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:24:29.115109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a1a74-17f2-4346-a332-cca626ba55ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:29.118530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48a1a74-17f2-4346-a332-cca626ba55ac admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:29.123346 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4109e-37af-4d8f-9476-96322abcdce5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5fc5cc3087f4ba68bba4daa437b7cc6, user_id=564bc620e923434b9389923314499d6a, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:29.129730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4109e-37af-4d8f-9476-96322abcdce5 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:29.165659 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1076/2135] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:24:29.170708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a1a74-17f2-4346-a332-cca626ba55ac admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:29.176033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1060/2136] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:29 2022] PUT /identity/v3/projects/a5fc5cc3087f4ba68bba4daa437b7cc6/users/564bc620e923434b9389923314499d6a/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:29.191184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fb918-45ec-4dfb-a58a-ebf0dc9fefa8 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:29.191559 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fb918-45ec-4dfb-a58a-ebf0dc9fefa8 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:29.191751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fb918-45ec-4dfb-a58a-ebf0dc9fefa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:29.195921 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a1a74-17f2-4346-a332-cca626ba55ac admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:29.259713 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-6f3fb918-45ec-4dfb-a58a-ebf0dc9fefa8 None None] MFA Rules not processed for user `564bc620e923434b9389923314499d6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:29.261960 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-6f3fb918-45ec-4dfb-a58a-ebf0dc9fefa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:29.314910 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a4099-c621-450f-8993-5248986a5968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:29.317934 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a4099-c621-450f-8993-5248986a5968 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:29.319937 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a4099-c621-450f-8993-5248986a5968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:29.320336 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a4099-c621-450f-8993-5248986a5968 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:29.321105 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a4099-c621-450f-8993-5248986a5968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:29.325284 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a4099-c621-450f-8993-5248986a5968 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:29.325850 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:29.364827 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:29.398524 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1077/2137] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:29 2022] POST /identity/v3/projects => generated 379 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:29.400058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a4099-c621-450f-8993-5248986a5968 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:29.420983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1061/2138] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:29.437393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fb918-45ec-4dfb-a58a-ebf0dc9fefa8 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:29.509620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fb918-45ec-4dfb-a58a-ebf0dc9fefa8 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:29.515893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a4099-c621-450f-8993-5248986a5968 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:29.539262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c227bdd-9728-46e9-bb8c-455652cd95b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:29.540054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c227bdd-9728-46e9-bb8c-455652cd95b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:29.540367 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c227bdd-9728-46e9-bb8c-455652cd95b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:29.540636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c227bdd-9728-46e9-bb8c-455652cd95b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:29.542844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-8c227bdd-9728-46e9-bb8c-455652cd95b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:29.657047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa632d1-0772-4371-8077-4a713049a2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:29.657958 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa632d1-0772-4371-8077-4a713049a2a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:29.658523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa632d1-0772-4371-8077-4a713049a2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:29.660022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c227bdd-9728-46e9-bb8c-455652cd95b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:29.660413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa632d1-0772-4371-8077-4a713049a2a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:29.661643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa632d1-0772-4371-8077-4a713049a2a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:29.665242 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c227bdd-9728-46e9-bb8c-455652cd95b4 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:29.670178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa632d1-0772-4371-8077-4a713049a2a4 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:29.690234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1078/2139] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:29.693083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c227bdd-9728-46e9-bb8c-455652cd95b4 service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:29.696274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-8c227bdd-9728-46e9-bb8c-455652cd95b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:29.725066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1062/2140] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:29 2022] POST /identity/v3/users => generated 290 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:29.727795 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa632d1-0772-4371-8077-4a713049a2a4 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:29.749261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa632d1-0772-4371-8077-4a713049a2a4 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:29.765731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207fb5f-45f1-40ff-be22-6a2ca69f52be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:29.766473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207fb5f-45f1-40ff-be22-6a2ca69f52be admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:29.768754 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207fb5f-45f1-40ff-be22-6a2ca69f52be admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:29.769739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207fb5f-45f1-40ff-be22-6a2ca69f52be admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:29.770516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207fb5f-45f1-40ff-be22-6a2ca69f52be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:29.773168 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207fb5f-45f1-40ff-be22-6a2ca69f52be admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:29.803125 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1063/2141] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:24:29.804871 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207fb5f-45f1-40ff-be22-6a2ca69f52be admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:29.829472 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207fb5f-45f1-40ff-be22-6a2ca69f52be admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:29.851476 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cc577-6028-463c-9ed0-5cecc3582419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:29.852459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cc577-6028-463c-9ed0-5cecc3582419 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:29.852752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cc577-6028-463c-9ed0-5cecc3582419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:29.855631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039cc577-6028-463c-9ed0-5cecc3582419 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:29.857869 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cc577-6028-463c-9ed0-5cecc3582419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:29.862063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039cc577-6028-463c-9ed0-5cecc3582419 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:29.899099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1064/2142] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:29.904751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cc577-6028-463c-9ed0-5cecc3582419 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:29.905857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c227bdd-9728-46e9-bb8c-455652cd95b4 service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:29.918246 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a3e71-17e0-464b-af8d-eec68a65ef8b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e08e00bcb5f47ffab58974ba376938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:29.921654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a3e71-17e0-464b-af8d-eec68a65ef8b service glance] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:29.922259 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a3e71-17e0-464b-af8d-eec68a65ef8b service glance] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:29.922624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a3e71-17e0-464b-af8d-eec68a65ef8b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:29.934724 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039cc577-6028-463c-9ed0-5cecc3582419 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:29.948569 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce828e7-dae1-4295-9bf5-f425526941cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:29.949238 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce828e7-dae1-4295-9bf5-f425526941cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:29.949529 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce828e7-dae1-4295-9bf5-f425526941cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:29.949946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce828e7-dae1-4295-9bf5-f425526941cf admin admin] PATH_INFO: `/v3/projects/d243a558d46347cbad332e46e5096940/users/7291c9528abb4ff49d225bfd1e1aa714/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:29.996337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce828e7-dae1-4295-9bf5-f425526941cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d243a558d46347cbad332e46e5096940, user_id=7291c9528abb4ff49d225bfd1e1aa714, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:29.999534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce828e7-dae1-4295-9bf5-f425526941cf admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:30.036824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1065/2143] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:29 2022] PUT /identity/v3/projects/d243a558d46347cbad332e46e5096940/users/7291c9528abb4ff49d225bfd1e1aa714/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:24:30.051435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7230274-3d15-4dab-9591-e38b81fe5b05 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:30.051435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7230274-3d15-4dab-9591-e38b81fe5b05 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:30.051435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7230274-3d15-4dab-9591-e38b81fe5b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:30.061374 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:30.108476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a3e71-17e0-464b-af8d-eec68a65ef8b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:30.111939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a3e71-17e0-464b-af8d-eec68a65ef8b service glance] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:30.127102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e7230274-3d15-4dab-9591-e38b81fe5b05 None None] MFA Rules not processed for user `7291c9528abb4ff49d225bfd1e1aa714`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:30.131212 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e7230274-3d15-4dab-9591-e38b81fe5b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:30.134958 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1079/2144] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:30.303912 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1066/2145] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:30.305881 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a3e71-17e0-464b-af8d-eec68a65ef8b service glance] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:30.327727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a3e71-17e0-464b-af8d-eec68a65ef8b service glance] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:30.396391 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef5017-4f85-4aef-8263-40553021e912 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:30.396896 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef5017-4f85-4aef-8263-40553021e912 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:30.397388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef5017-4f85-4aef-8263-40553021e912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:30.422866 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:30.470918 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-1eef5017-4f85-4aef-8263-40553021e912 None None] MFA Rules not processed for user `2ab18214df00412bb7440535ac725056`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:30.474711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1eef5017-4f85-4aef-8263-40553021e912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:30.534412 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222d503-a2e9-4120-999e-47964bc5a107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:30.534963 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222d503-a2e9-4120-999e-47964bc5a107 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:30.535248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222d503-a2e9-4120-999e-47964bc5a107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:30.535621 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222d503-a2e9-4120-999e-47964bc5a107 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:30.536265 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222d503-a2e9-4120-999e-47964bc5a107 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:30.538210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222d503-a2e9-4120-999e-47964bc5a107 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:30.600918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1080/2146] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:30 2022] POST /identity/v3/projects => generated 375 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:30.605028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222d503-a2e9-4120-999e-47964bc5a107 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:30.620121 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1067/2147] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:30.621424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef5017-4f85-4aef-8263-40553021e912 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:30.624393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1eef5017-4f85-4aef-8263-40553021e912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:30.669467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef5017-4f85-4aef-8263-40553021e912 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:30.757085 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222d503-a2e9-4120-999e-47964bc5a107 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:30.767444 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649705f-e34d-45e3-b791-bd214a066e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:30.768080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649705f-e34d-45e3-b791-bd214a066e8c admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:30.770110 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649705f-e34d-45e3-b791-bd214a066e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:30.770110 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649705f-e34d-45e3-b791-bd214a066e8c admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:30.770763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649705f-e34d-45e3-b791-bd214a066e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:30.772707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649705f-e34d-45e3-b791-bd214a066e8c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:30.775947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604cacc8-409f-4fb4-9486-ac77f82d35a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:30.776585 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604cacc8-409f-4fb4-9486-ac77f82d35a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:30.776949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604cacc8-409f-4fb4-9486-ac77f82d35a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:30.777369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604cacc8-409f-4fb4-9486-ac77f82d35a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:30.778690 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604cacc8-409f-4fb4-9486-ac77f82d35a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:30.781319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604cacc8-409f-4fb4-9486-ac77f82d35a6 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:30.830425 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1081/2148] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:30 2022] POST /identity/v3/users => generated 288 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:30.837456 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649705f-e34d-45e3-b791-bd214a066e8c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:30.857429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649705f-e34d-45e3-b791-bd214a066e8c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:30.860633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1068/2149] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:30 2022] POST /identity/v3/projects => generated 385 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:30.865507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604cacc8-409f-4fb4-9486-ac77f82d35a6 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:30.999197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:31.012740 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502d42b-a6f3-4dad-9a7f-63fb24dfa144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:31.013469 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502d42b-a6f3-4dad-9a7f-63fb24dfa144 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:31.014356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502d42b-a6f3-4dad-9a7f-63fb24dfa144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:31.014356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502d42b-a6f3-4dad-9a7f-63fb24dfa144 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:31.014929 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502d42b-a6f3-4dad-9a7f-63fb24dfa144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:31.016402 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502d42b-a6f3-4dad-9a7f-63fb24dfa144 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:31.036885 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604cacc8-409f-4fb4-9486-ac77f82d35a6 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:31.044342 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1082/2150] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:31.045975 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502d42b-a6f3-4dad-9a7f-63fb24dfa144 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:31.051791 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb168c-7b30-4806-9e89-1c6332def9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:31.052416 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb168c-7b30-4806-9e89-1c6332def9df admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:31.052708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb168c-7b30-4806-9e89-1c6332def9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:31.052986 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb168c-7b30-4806-9e89-1c6332def9df admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:31.054495 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb168c-7b30-4806-9e89-1c6332def9df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:31.057250 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb168c-7b30-4806-9e89-1c6332def9df admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:31.067887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502d42b-a6f3-4dad-9a7f-63fb24dfa144 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:31.085705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e08736-d33f-4326-bad1-29c09e33bcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:31.085705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e08736-d33f-4326-bad1-29c09e33bcf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:31.085705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e08736-d33f-4326-bad1-29c09e33bcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:31.085705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e08736-d33f-4326-bad1-29c09e33bcf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:31.086240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e08736-d33f-4326-bad1-29c09e33bcf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:31.090473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e08736-d33f-4326-bad1-29c09e33bcf9 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:31.118500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1069/2151] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:30 2022] POST /identity/v3/users => generated 299 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:31.118700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1083/2152] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:31.123026 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb168c-7b30-4806-9e89-1c6332def9df admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:31.124855 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e08736-d33f-4326-bad1-29c09e33bcf9 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:31.136690 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e08736-d33f-4326-bad1-29c09e33bcf9 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:31.142398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba522e9b-28f8-4ae1-9810-510c3b34cce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:31.142818 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb168c-7b30-4806-9e89-1c6332def9df admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:31.143020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba522e9b-28f8-4ae1-9810-510c3b34cce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:31.143344 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba522e9b-28f8-4ae1-9810-510c3b34cce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:31.143618 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba522e9b-28f8-4ae1-9810-510c3b34cce8 admin admin] PATH_INFO: `/v3/projects/592de5a63f11449ca52b1961cb224bf0/users/fac0ceba35b5469489a22035b5be1839/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:31.163921 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f39686-e1df-4a37-8b42-1386c112ec11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:31.164556 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f39686-e1df-4a37-8b42-1386c112ec11 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:31.164838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f39686-e1df-4a37-8b42-1386c112ec11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:31.170335 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f39686-e1df-4a37-8b42-1386c112ec11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:31.170335 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f39686-e1df-4a37-8b42-1386c112ec11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:31.170335 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f39686-e1df-4a37-8b42-1386c112ec11 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:31.179041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba522e9b-28f8-4ae1-9810-510c3b34cce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=592de5a63f11449ca52b1961cb224bf0, user_id=fac0ceba35b5469489a22035b5be1839, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:31.181973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:31.185384 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba522e9b-28f8-4ae1-9810-510c3b34cce8 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:31.193615 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1070/2153] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:31.197670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f39686-e1df-4a37-8b42-1386c112ec11 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:31.214951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f39686-e1df-4a37-8b42-1386c112ec11 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:31.232992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1084/2154] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:31 2022] PUT /identity/v3/projects/592de5a63f11449ca52b1961cb224bf0/users/fac0ceba35b5469489a22035b5be1839/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:31.245868 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de8989-ee8c-447a-b83c-dc5c77800b50 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:31.246171 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de8989-ee8c-447a-b83c-dc5c77800b50 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:31.246447 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de8989-ee8c-447a-b83c-dc5c77800b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:31.308275 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-73de8989-ee8c-447a-b83c-dc5c77800b50 None None] MFA Rules not processed for user `fac0ceba35b5469489a22035b5be1839`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:31.309589 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-73de8989-ee8c-447a-b83c-dc5c77800b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:31.336611 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4db25f-234d-415c-8467-e188fc496fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:31.338329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4db25f-234d-415c-8467-e188fc496fea admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:31.338500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4db25f-234d-415c-8467-e188fc496fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:31.338766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4db25f-234d-415c-8467-e188fc496fea admin admin] PATH_INFO: `/v3/projects/6131a9e67e1b4ac2ad3d28f2fad23f67/users/1d44cd6ef5824545afafcb831fd1be32/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:31.376984 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4db25f-234d-415c-8467-e188fc496fea admin admin] RBAC: Authorizing `identity:create_grant(project_id=6131a9e67e1b4ac2ad3d28f2fad23f67, user_id=1d44cd6ef5824545afafcb831fd1be32, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:31.386184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4db25f-234d-415c-8467-e188fc496fea admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:31.417123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1071/2155] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:31 2022] PUT /identity/v3/projects/6131a9e67e1b4ac2ad3d28f2fad23f67/users/1d44cd6ef5824545afafcb831fd1be32/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:24:31.418073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4db25f-234d-415c-8467-e188fc496fea admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:31.453058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1085/2156] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:31.459297 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de8989-ee8c-447a-b83c-dc5c77800b50 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:31.483790 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de8989-ee8c-447a-b83c-dc5c77800b50 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:31.554440 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4db25f-234d-415c-8467-e188fc496fea admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:31.559521 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08244012-3100-4958-8ae1-94aecbaa635c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:31.560453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08244012-3100-4958-8ae1-94aecbaa635c admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:31.560453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08244012-3100-4958-8ae1-94aecbaa635c admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:31.560966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08244012-3100-4958-8ae1-94aecbaa635c admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:31.561809 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08244012-3100-4958-8ae1-94aecbaa635c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:31.563445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08244012-3100-4958-8ae1-94aecbaa635c admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:31.577358 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1072/2157] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:24:31.586223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08244012-3100-4958-8ae1-94aecbaa635c admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:31.593648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494695d3-874d-42e4-8d1d-06daf0ec698c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:31.594452 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494695d3-874d-42e4-8d1d-06daf0ec698c admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:31.594947 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494695d3-874d-42e4-8d1d-06daf0ec698c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:31.595520 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494695d3-874d-42e4-8d1d-06daf0ec698c admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:31.601679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494695d3-874d-42e4-8d1d-06daf0ec698c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:31.611327 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08244012-3100-4958-8ae1-94aecbaa635c admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:31.614710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494695d3-874d-42e4-8d1d-06daf0ec698c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:31.624905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b99a65-3862-40e2-ab2e-784ab4080c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:31.626225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b99a65-3862-40e2-ab2e-784ab4080c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:31.626885 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b99a65-3862-40e2-ab2e-784ab4080c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:31.627287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b99a65-3862-40e2-ab2e-784ab4080c84 admin admin] PATH_INFO: `/v3/projects/6131a9e67e1b4ac2ad3d28f2fad23f67/users/1d44cd6ef5824545afafcb831fd1be32/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:31.634736 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b99a65-3862-40e2-ab2e-784ab4080c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6131a9e67e1b4ac2ad3d28f2fad23f67, user_id=1d44cd6ef5824545afafcb831fd1be32, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:31.641047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b99a65-3862-40e2-ab2e-784ab4080c84 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:31.687088 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1073/2158] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:31 2022] PUT /identity/v3/projects/6131a9e67e1b4ac2ad3d28f2fad23f67/users/1d44cd6ef5824545afafcb831fd1be32/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:31.726105 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1086/2159] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:31 2022] POST /identity/v3/projects => generated 379 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:31.730252 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b99a65-3862-40e2-ab2e-784ab4080c84 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:31.929157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b99a65-3862-40e2-ab2e-784ab4080c84 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:31.942607 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:32.045771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820d161-1b8a-449f-a623-6c77770a35a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:32.046411 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820d161-1b8a-449f-a623-6c77770a35a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:32.046699 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820d161-1b8a-449f-a623-6c77770a35a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:32.046979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820d161-1b8a-449f-a623-6c77770a35a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:32.047626 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820d161-1b8a-449f-a623-6c77770a35a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:32.049959 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820d161-1b8a-449f-a623-6c77770a35a3 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:32.095912 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1074/2160] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:31 2022] POST /identity/v3/users => generated 296 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:32.101122 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494695d3-874d-42e4-8d1d-06daf0ec698c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:32.119936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494695d3-874d-42e4-8d1d-06daf0ec698c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:32.134402 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f47bbb-c7c1-4735-9a1a-478f09b0ea65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:32.135115 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47bbb-c7c1-4735-9a1a-478f09b0ea65 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:32.135774 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47bbb-c7c1-4735-9a1a-478f09b0ea65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:32.137165 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f47bbb-c7c1-4735-9a1a-478f09b0ea65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:32.138809 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f47bbb-c7c1-4735-9a1a-478f09b0ea65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:32.143033 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f47bbb-c7c1-4735-9a1a-478f09b0ea65 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:32.173090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1087/2161] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:32.178080 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820d161-1b8a-449f-a623-6c77770a35a3 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:32.200967 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820d161-1b8a-449f-a623-6c77770a35a3 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:32.207634 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baefe45-0326-42b3-9551-c20c26eef1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:32.208520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baefe45-0326-42b3-9551-c20c26eef1a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:32.209020 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baefe45-0326-42b3-9551-c20c26eef1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:32.210180 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baefe45-0326-42b3-9551-c20c26eef1a2 admin admin] PATH_INFO: `/v3/projects/abc91942844f4bd2bbe13ceaf3abc071/users/fac090c2bc724e1096c74fb729e14f48/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:32.233590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baefe45-0326-42b3-9551-c20c26eef1a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abc91942844f4bd2bbe13ceaf3abc071, user_id=fac090c2bc724e1096c74fb729e14f48, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:32.239332 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baefe45-0326-42b3-9551-c20c26eef1a2 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:32.263272 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1075/2162] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:32 2022] PUT /identity/v3/projects/abc91942844f4bd2bbe13ceaf3abc071/users/fac090c2bc724e1096c74fb729e14f48/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:32.267166 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f47bbb-c7c1-4735-9a1a-478f09b0ea65 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:32.287096 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f47bbb-c7c1-4735-9a1a-478f09b0ea65 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:32.326575 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:32.386159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab6248-0cf2-4c96-bb45-bf6f1e465c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:32.386965 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab6248-0cf2-4c96-bb45-bf6f1e465c90 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:32.387345 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab6248-0cf2-4c96-bb45-bf6f1e465c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:32.387736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab6248-0cf2-4c96-bb45-bf6f1e465c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:32.388601 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab6248-0cf2-4c96-bb45-bf6f1e465c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:32.390588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab6248-0cf2-4c96-bb45-bf6f1e465c90 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:32.407373 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1088/2163] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:32.409226 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baefe45-0326-42b3-9551-c20c26eef1a2 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:32.419560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baefe45-0326-42b3-9551-c20c26eef1a2 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:32.452595 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093dc93-ca38-405f-8073-43e68c48f3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:32.453359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093dc93-ca38-405f-8073-43e68c48f3a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:32.453639 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093dc93-ca38-405f-8073-43e68c48f3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:32.453910 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8093dc93-ca38-405f-8073-43e68c48f3a4 admin admin] PATH_INFO: `/v3/projects/abc91942844f4bd2bbe13ceaf3abc071/users/fac090c2bc724e1096c74fb729e14f48/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:32.458295 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093dc93-ca38-405f-8073-43e68c48f3a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abc91942844f4bd2bbe13ceaf3abc071, user_id=fac090c2bc724e1096c74fb729e14f48, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:32.460980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8093dc93-ca38-405f-8073-43e68c48f3a4 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:32.500357 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51435a39-9cc6-4ddc-a4b5-d6e015ab024f None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:32.501978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51435a39-9cc6-4ddc-a4b5-d6e015ab024f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:32.502064 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1076/2164] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:32 2022] PUT /identity/v3/projects/abc91942844f4bd2bbe13ceaf3abc071/users/fac090c2bc724e1096c74fb729e14f48/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:32.502350 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51435a39-9cc6-4ddc-a4b5-d6e015ab024f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:32.572794 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-51435a39-9cc6-4ddc-a4b5-d6e015ab024f None None] MFA Rules not processed for user `fac090c2bc724e1096c74fb729e14f48`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:32.574074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-51435a39-9cc6-4ddc-a4b5-d6e015ab024f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:32.705007 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1089/2165] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:32.716287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093dc93-ca38-405f-8073-43e68c48f3a4 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:32.719751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8093dc93-ca38-405f-8073-43e68c48f3a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:32.824324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8093dc93-ca38-405f-8073-43e68c48f3a4 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:32.839281 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96fcfc-71c9-4501-97ad-b614fdb382fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:32.839759 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96fcfc-71c9-4501-97ad-b614fdb382fe service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:32.840044 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96fcfc-71c9-4501-97ad-b614fdb382fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:32.840646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96fcfc-71c9-4501-97ad-b614fdb382fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:32.865667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96fcfc-71c9-4501-97ad-b614fdb382fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:32.869479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96fcfc-71c9-4501-97ad-b614fdb382fe service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:32.890435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1077/2166] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:24:32.962485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51435a39-9cc6-4ddc-a4b5-d6e015ab024f None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:32.979546 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51435a39-9cc6-4ddc-a4b5-d6e015ab024f None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:32.989708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03213882-849d-4f28-ae20-9b64614682f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:32.992493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03213882-849d-4f28-ae20-9b64614682f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:32.992981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03213882-849d-4f28-ae20-9b64614682f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:32.993610 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03213882-849d-4f28-ae20-9b64614682f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:33.118780 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03213882-849d-4f28-ae20-9b64614682f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:33.122540 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03213882-849d-4f28-ae20-9b64614682f6 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:33.143180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1090/2167] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:34.437784 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe76f4-8fc8-4924-bfab-1e3e7c8dedb9 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:34.438248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe76f4-8fc8-4924-bfab-1e3e7c8dedb9 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:34.438338 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe76f4-8fc8-4924-bfab-1e3e7c8dedb9 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:34.439682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1078/2168] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 12:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:24:34.450293 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99232739-1816-4011-81df-7a39d349ebd3 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:34.451753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99232739-1816-4011-81df-7a39d349ebd3 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:34.452285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99232739-1816-4011-81df-7a39d349ebd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:34.528781 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-99232739-1816-4011-81df-7a39d349ebd3 None None] MFA Rules not processed for user `9a2322b1a76d40e49545f5677e247489`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:34.532723 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-99232739-1816-4011-81df-7a39d349ebd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:34.561796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe76f4-8fc8-4924-bfab-1e3e7c8dedb9 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:34.563879 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d4fe76f4-8fc8-4924-bfab-1e3e7c8dedb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:34.627328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:34.701216 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1091/2169] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 12:24:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:34.713446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99232739-1816-4011-81df-7a39d349ebd3 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:34.715236 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-99232739-1816-4011-81df-7a39d349ebd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:34.717803 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe76f4-8fc8-4924-bfab-1e3e7c8dedb9 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:34.730005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25df213-1776-477f-a37b-da7755fa09bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:34.731870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25df213-1776-477f-a37b-da7755fa09bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:34.732163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25df213-1776-477f-a37b-da7755fa09bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:34.732446 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25df213-1776-477f-a37b-da7755fa09bd admin admin] PATH_INFO: `/v3/users/95575c3357d4413298c3fa08709c5037` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:34.733985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25df213-1776-477f-a37b-da7755fa09bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=95575c3357d4413298c3fa08709c5037)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:34.736722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25df213-1776-477f-a37b-da7755fa09bd admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:34.739742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99232739-1816-4011-81df-7a39d349ebd3 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:34.750805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53592d04-da88-4e6f-b339-beb0e74ee4a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:34.751882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:34.754969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53592d04-da88-4e6f-b339-beb0e74ee4a2 service placement] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:34.756901 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53592d04-da88-4e6f-b339-beb0e74ee4a2 service placement] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:34.758468 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53592d04-da88-4e6f-b339-beb0e74ee4a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:34.783726 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53592d04-da88-4e6f-b339-beb0e74ee4a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:34.790541 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53592d04-da88-4e6f-b339-beb0e74ee4a2 service placement] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:34.809041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1092/2170] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:34.834983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-e25df213-1776-477f-a37b-da7755fa09bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95575c3357d4413298c3fa08709c5037'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:34.835805 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-e25df213-1776-477f-a37b-da7755fa09bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95575c3357d4413298c3fa08709c5037'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:34.862640 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-e25df213-1776-477f-a37b-da7755fa09bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95575c3357d4413298c3fa08709c5037'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:34.887607 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-e25df213-1776-477f-a37b-da7755fa09bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95575c3357d4413298c3fa08709c5037'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:34.888350 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-e25df213-1776-477f-a37b-da7755fa09bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95575c3357d4413298c3fa08709c5037'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:34.918898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1079/2171] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:34 2022] DELETE /identity/v3/users/95575c3357d4413298c3fa08709c5037 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:35.200363 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53592d04-da88-4e6f-b339-beb0e74ee4a2 service placement] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:35.217322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53592d04-da88-4e6f-b339-beb0e74ee4a2 service placement] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:35.319890 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb57234-b7fb-4a77-b818-e30edee79d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:35.320653 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb57234-b7fb-4a77-b818-e30edee79d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:35.321006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb57234-b7fb-4a77-b818-e30edee79d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:35.321406 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb57234-b7fb-4a77-b818-e30edee79d0e admin admin] PATH_INFO: `/v3/projects/3afaa30b2f454bc0bf1f23ece426339b` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:35.322682 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb57234-b7fb-4a77-b818-e30edee79d0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3afaa30b2f454bc0bf1f23ece426339b)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:35.324735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb57234-b7fb-4a77-b818-e30edee79d0e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:35.398385 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-bdb57234-b7fb-4a77-b818-e30edee79d0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3afaa30b2f454bc0bf1f23ece426339b', 'request_id': 'req-bdb57234-b7fb-4a77-b818-e30edee79d0e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:35.407331 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-bdb57234-b7fb-4a77-b818-e30edee79d0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3afaa30b2f454bc0bf1f23ece426339b', 'request_id': 'req-bdb57234-b7fb-4a77-b818-e30edee79d0e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:35.424771 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-bdb57234-b7fb-4a77-b818-e30edee79d0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3afaa30b2f454bc0bf1f23ece426339b', 'request_id': 'req-bdb57234-b7fb-4a77-b818-e30edee79d0e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:35.462371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-bdb57234-b7fb-4a77-b818-e30edee79d0e admin admin] The token cache is being invalidate because project 3afaa30b2f454bc0bf1f23ece426339b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:35.463874 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-bdb57234-b7fb-4a77-b818-e30edee79d0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:35.464819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-bdb57234-b7fb-4a77-b818-e30edee79d0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:35.467961 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1093/2172] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:35 2022] DELETE /identity/v3/projects/3afaa30b2f454bc0bf1f23ece426339b => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:35.473848 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5bc94-f31a-48c3-8b9c-641e118efc69 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:35.474727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5bc94-f31a-48c3-8b9c-641e118efc69 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:35.475056 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5bc94-f31a-48c3-8b9c-641e118efc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:35.516420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-f0a5bc94-f31a-48c3-8b9c-641e118efc69 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:35.518131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f0a5bc94-f31a-48c3-8b9c-641e118efc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:35.658303 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1080/2173] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:35.666158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdcaa2-ca1b-4793-bd73-736147ce2560 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:35.666477 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdcaa2-ca1b-4793-bd73-736147ce2560 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:35.666766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdcaa2-ca1b-4793-bd73-736147ce2560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:35.700773 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-e0cdcaa2-ca1b-4793-bd73-736147ce2560 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:35.702729 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e0cdcaa2-ca1b-4793-bd73-736147ce2560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:35.712100 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1094/2174] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:35.716099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5bc94-f31a-48c3-8b9c-641e118efc69 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:35.728505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5bc94-f31a-48c3-8b9c-641e118efc69 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:35.733786 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ebe2d-a12f-4ef0-b2f4-84c4e9704cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:35.734773 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ebe2d-a12f-4ef0-b2f4-84c4e9704cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:35.735035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ebe2d-a12f-4ef0-b2f4-84c4e9704cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:35.735410 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ebe2d-a12f-4ef0-b2f4-84c4e9704cfc admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:35.736182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ebe2d-a12f-4ef0-b2f4-84c4e9704cfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:24:35.736592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ebe2d-a12f-4ef0-b2f4-84c4e9704cfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:35.739438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ebe2d-a12f-4ef0-b2f4-84c4e9704cfc admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:35.752606 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1081/2175] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:24:35.757726 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdcaa2-ca1b-4793-bd73-736147ce2560 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:35.770287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdcaa2-ca1b-4793-bd73-736147ce2560 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:35.777259 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740cbba-1338-4e73-9074-22eb0c725c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:35.778020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740cbba-1338-4e73-9074-22eb0c725c5e admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:35.778404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740cbba-1338-4e73-9074-22eb0c725c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:35.778800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740cbba-1338-4e73-9074-22eb0c725c5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:35.779554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740cbba-1338-4e73-9074-22eb0c725c5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:35.781644 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740cbba-1338-4e73-9074-22eb0c725c5e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:35.826505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1095/2176] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:35 2022] POST /identity/v3/projects => generated 391 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:35.826720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ebe2d-a12f-4ef0-b2f4-84c4e9704cfc admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:35.839676 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ebe2d-a12f-4ef0-b2f4-84c4e9704cfc admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:35.915490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ff087-f491-4312-92c1-9bba44bce006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:35.916346 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ff087-f491-4312-92c1-9bba44bce006 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:35.916635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ff087-f491-4312-92c1-9bba44bce006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:35.916965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ff087-f491-4312-92c1-9bba44bce006 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:35.917703 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ff087-f491-4312-92c1-9bba44bce006 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:35.919633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ff087-f491-4312-92c1-9bba44bce006 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:35.944558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1082/2177] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:35 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:35.949210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740cbba-1338-4e73-9074-22eb0c725c5e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:35.962663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740cbba-1338-4e73-9074-22eb0c725c5e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:35.974948 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ce75c-4002-4a6f-94a4-f9babd45532c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:35.975690 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ce75c-4002-4a6f-94a4-f9babd45532c admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:35.976072 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ce75c-4002-4a6f-94a4-f9babd45532c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:35.976448 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ce75c-4002-4a6f-94a4-f9babd45532c admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:35.977243 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ce75c-4002-4a6f-94a4-f9babd45532c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:35.979137 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ce75c-4002-4a6f-94a4-f9babd45532c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:35.992618 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1096/2178] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:35.996889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ff087-f491-4312-92c1-9bba44bce006 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:36.004600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:36.018954 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ff087-f491-4312-92c1-9bba44bce006 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:36.029830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531d116-cd78-4ca9-88c7-47fa6d86fee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:36.030479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531d116-cd78-4ca9-88c7-47fa6d86fee6 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:36.031959 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531d116-cd78-4ca9-88c7-47fa6d86fee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:36.032247 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531d116-cd78-4ca9-88c7-47fa6d86fee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:36.033011 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531d116-cd78-4ca9-88c7-47fa6d86fee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:36.040547 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531d116-cd78-4ca9-88c7-47fa6d86fee6 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:36.055780 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1083/2179] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:36.059826 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ce75c-4002-4a6f-94a4-f9babd45532c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:36.075893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ce75c-4002-4a6f-94a4-f9babd45532c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:36.080980 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5921d5-d9ed-4c12-afbe-ea4b6eba331c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:36.081955 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5921d5-d9ed-4c12-afbe-ea4b6eba331c admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:36.082302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5921d5-d9ed-4c12-afbe-ea4b6eba331c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:36.082527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5921d5-d9ed-4c12-afbe-ea4b6eba331c admin admin] PATH_INFO: `/v3/projects/1318535cbbd6427c8c770bd5b256ac3e/users/cc96808ff3fb41f085f09aa943c3d016/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:36.095606 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:36.101247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5921d5-d9ed-4c12-afbe-ea4b6eba331c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1318535cbbd6427c8c770bd5b256ac3e, user_id=cc96808ff3fb41f085f09aa943c3d016, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:36.104446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5921d5-d9ed-4c12-afbe-ea4b6eba331c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:36.140123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1097/2180] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:36 2022] PUT /identity/v3/projects/1318535cbbd6427c8c770bd5b256ac3e/users/cc96808ff3fb41f085f09aa943c3d016/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:36.146822 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a747d-bbec-405f-8026-029f88134cff None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:36.147110 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a747d-bbec-405f-8026-029f88134cff None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:36.147385 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a747d-bbec-405f-8026-029f88134cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:36.199626 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-d45a747d-bbec-405f-8026-029f88134cff None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:36.200913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d45a747d-bbec-405f-8026-029f88134cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:36.321501 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1084/2181] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:36.332592 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5921d5-d9ed-4c12-afbe-ea4b6eba331c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:36.336222 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-ca5921d5-d9ed-4c12-afbe-ea4b6eba331c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:36.411149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5921d5-d9ed-4c12-afbe-ea4b6eba331c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:36.415252 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd578d-9d80-4845-9a99-b5b50189007a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:36.415877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd578d-9d80-4845-9a99-b5b50189007a service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:36.416300 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd578d-9d80-4845-9a99-b5b50189007a service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:36.416576 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd578d-9d80-4845-9a99-b5b50189007a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:36.430819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd578d-9d80-4845-9a99-b5b50189007a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:36.431922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd578d-9d80-4845-9a99-b5b50189007a service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:36.452543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1098/2182] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:38.717740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fe856-37e2-42b7-bceb-faa923b6b9d3 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:38.718203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fe856-37e2-42b7-bceb-faa923b6b9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:38.718203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815fe856-37e2-42b7-bceb-faa923b6b9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:38.761788 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-815fe856-37e2-42b7-bceb-faa923b6b9d3 None None] MFA Rules not processed for user `1d44cd6ef5824545afafcb831fd1be32`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:38.763286 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-815fe856-37e2-42b7-bceb-faa923b6b9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:38.862732 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1085/2183] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:38.880501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd578d-9d80-4845-9a99-b5b50189007a service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:38.883858 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-fddd578d-9d80-4845-9a99-b5b50189007a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:38.945041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd578d-9d80-4845-9a99-b5b50189007a service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:38.948835 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb941d-cd35-4d39-84e1-5f8b552db17e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e08e00bcb5f47ffab58974ba376938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:38.949493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb941d-cd35-4d39-84e1-5f8b552db17e service glance] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:38.949771 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb941d-cd35-4d39-84e1-5f8b552db17e service glance] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:38.950041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb941d-cd35-4d39-84e1-5f8b552db17e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:38.951858 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-cefb941d-cd35-4d39-84e1-5f8b552db17e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:39.025520 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb941d-cd35-4d39-84e1-5f8b552db17e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:39.026858 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb941d-cd35-4d39-84e1-5f8b552db17e service glance] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:39.040020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1099/2184] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:39.215547 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3741951-ebf7-4606-b29f-dc3a4c5aa3b9 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:39.215822 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3741951-ebf7-4606-b29f-dc3a4c5aa3b9 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:39.216119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3741951-ebf7-4606-b29f-dc3a4c5aa3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:39.247543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-c3741951-ebf7-4606-b29f-dc3a4c5aa3b9 None None] MFA Rules not processed for user `bc69cf267bea41bf890bc7551d84ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:39.248814 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c3741951-ebf7-4606-b29f-dc3a4c5aa3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:39.370654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1086/2185] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 12:24:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:39.434288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb941d-cd35-4d39-84e1-5f8b552db17e service glance] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:39.451997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-cefb941d-cd35-4d39-84e1-5f8b552db17e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:39.563210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:39.591390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb941d-cd35-4d39-84e1-5f8b552db17e service glance] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:39.597170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48872470-ad18-4cf1-832b-d8677d60e48e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975a8d7349994520bcbfb5f75e52aac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:39.598263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48872470-ad18-4cf1-832b-d8677d60e48e service swift] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:39.598643 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48872470-ad18-4cf1-832b-d8677d60e48e service swift] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:39.599160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48872470-ad18-4cf1-832b-d8677d60e48e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:39.609473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48872470-ad18-4cf1-832b-d8677d60e48e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:39.611191 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48872470-ad18-4cf1-832b-d8677d60e48e service swift] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:39.629659 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1100/2186] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 12:24:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 12:24:42.988213 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb31e8-846b-4413-940c-8af5e6c00eae None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:42.988213 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb31e8-846b-4413-940c-8af5e6c00eae None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:42.988845 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb31e8-846b-4413-940c-8af5e6c00eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:43.062965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-7cfb31e8-846b-4413-940c-8af5e6c00eae None None] MFA Rules not processed for user `cc96808ff3fb41f085f09aa943c3d016`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:43.064204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7cfb31e8-846b-4413-940c-8af5e6c00eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:43.111547 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:43.277398 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1087/2187] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:43.293187 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5df30c-60b7-415e-88dd-6d914fc6d0b1 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:43.294331 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5df30c-60b7-415e-88dd-6d914fc6d0b1 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:43.294848 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5df30c-60b7-415e-88dd-6d914fc6d0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:43.333377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-3d5df30c-60b7-415e-88dd-6d914fc6d0b1 None None] MFA Rules not processed for user `cc96808ff3fb41f085f09aa943c3d016`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:43.335148 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-3d5df30c-60b7-415e-88dd-6d914fc6d0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:43.360322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1101/2188] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:43.371382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb31e8-846b-4413-940c-8af5e6c00eae None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:43.372632 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7cfb31e8-846b-4413-940c-8af5e6c00eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:43.522722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb31e8-846b-4413-940c-8af5e6c00eae None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:43.537165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c632e6f-ac37-4158-97c9-566162d08bb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:43.541029 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c632e6f-ac37-4158-97c9-566162d08bb6 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:43.541565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c632e6f-ac37-4158-97c9-566162d08bb6 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:43.541910 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c632e6f-ac37-4158-97c9-566162d08bb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:43.555647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c632e6f-ac37-4158-97c9-566162d08bb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:43.557570 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c632e6f-ac37-4158-97c9-566162d08bb6 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:43.579274 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1088/2189] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:44.970356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5df30c-60b7-415e-88dd-6d914fc6d0b1 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:44.972370 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-3d5df30c-60b7-415e-88dd-6d914fc6d0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:44.995439 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5df30c-60b7-415e-88dd-6d914fc6d0b1 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:45.000981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91370cce-60db-4854-aade-b5e26c49e5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:45.002129 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91370cce-60db-4854-aade-b5e26c49e5f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:45.002762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91370cce-60db-4854-aade-b5e26c49e5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:45.003198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91370cce-60db-4854-aade-b5e26c49e5f3 admin admin] PATH_INFO: `/v3/users/7291c9528abb4ff49d225bfd1e1aa714` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:45.004583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91370cce-60db-4854-aade-b5e26c49e5f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7291c9528abb4ff49d225bfd1e1aa714)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:45.006701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91370cce-60db-4854-aade-b5e26c49e5f3 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:45.101705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-91370cce-60db-4854-aade-b5e26c49e5f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7291c9528abb4ff49d225bfd1e1aa714'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:45.102593 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-91370cce-60db-4854-aade-b5e26c49e5f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7291c9528abb4ff49d225bfd1e1aa714'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:45.113418 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-91370cce-60db-4854-aade-b5e26c49e5f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7291c9528abb4ff49d225bfd1e1aa714'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:45.114109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-91370cce-60db-4854-aade-b5e26c49e5f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7291c9528abb4ff49d225bfd1e1aa714'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:45.122664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-91370cce-60db-4854-aade-b5e26c49e5f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7291c9528abb4ff49d225bfd1e1aa714'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:45.158452 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bac289-71cd-41df-a62e-3df797fb6738 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:45.160192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bac289-71cd-41df-a62e-3df797fb6738 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:45.160711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bac289-71cd-41df-a62e-3df797fb6738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:45.162268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1102/2190] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:44 2022] DELETE /identity/v3/users/7291c9528abb4ff49d225bfd1e1aa714 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:24:45.205125 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-11bac289-71cd-41df-a62e-3df797fb6738 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:45.207256 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-11bac289-71cd-41df-a62e-3df797fb6738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:45.310493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1089/2191] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:45.317324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91370cce-60db-4854-aade-b5e26c49e5f3 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:45.318722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-91370cce-60db-4854-aade-b5e26c49e5f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:45.423309 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91370cce-60db-4854-aade-b5e26c49e5f3 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:45.427903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8947ce8-25ce-4c2b-988a-49421576ffe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:45.428590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8947ce8-25ce-4c2b-988a-49421576ffe5 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:45.428876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8947ce8-25ce-4c2b-988a-49421576ffe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:45.429238 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8947ce8-25ce-4c2b-988a-49421576ffe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:45.442275 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8947ce8-25ce-4c2b-988a-49421576ffe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:45.443938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8947ce8-25ce-4c2b-988a-49421576ffe5 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:45.458523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1103/2192] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:45.761978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bac289-71cd-41df-a62e-3df797fb6738 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:45.764284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-11bac289-71cd-41df-a62e-3df797fb6738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:45.800595 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bac289-71cd-41df-a62e-3df797fb6738 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:45.817059 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d86404-4db6-4e9e-85b0-5ea5d3eeb64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:45.825465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d86404-4db6-4e9e-85b0-5ea5d3eeb64a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:45.825966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d86404-4db6-4e9e-85b0-5ea5d3eeb64a admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:45.826448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d86404-4db6-4e9e-85b0-5ea5d3eeb64a admin admin] PATH_INFO: `/v3/users/be8b051080db459d9818738fedb4a913` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:45.833310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d86404-4db6-4e9e-85b0-5ea5d3eeb64a admin admin] RBAC: Authorizing `identity:delete_user(user_id=be8b051080db459d9818738fedb4a913)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:45.835497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d86404-4db6-4e9e-85b0-5ea5d3eeb64a admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:45.952998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:45.981771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-43d86404-4db6-4e9e-85b0-5ea5d3eeb64a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be8b051080db459d9818738fedb4a913'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:45.984837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-43d86404-4db6-4e9e-85b0-5ea5d3eeb64a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be8b051080db459d9818738fedb4a913'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:46.037300 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-43d86404-4db6-4e9e-85b0-5ea5d3eeb64a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be8b051080db459d9818738fedb4a913'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:46.049118 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-43d86404-4db6-4e9e-85b0-5ea5d3eeb64a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be8b051080db459d9818738fedb4a913'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:46.050657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-43d86404-4db6-4e9e-85b0-5ea5d3eeb64a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be8b051080db459d9818738fedb4a913'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:46.064075 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8947ce8-25ce-4c2b-988a-49421576ffe5 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:46.069643 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b8947ce8-25ce-4c2b-988a-49421576ffe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:46.087422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8947ce8-25ce-4c2b-988a-49421576ffe5 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:46.102589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1090/2193] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:45 2022] DELETE /identity/v3/users/be8b051080db459d9818738fedb4a913 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:46.104320 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a29624-10c2-4dae-a063-fb06358f8963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:46.104963 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a29624-10c2-4dae-a063-fb06358f8963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:46.105350 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a29624-10c2-4dae-a063-fb06358f8963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:46.105789 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a29624-10c2-4dae-a063-fb06358f8963 admin admin] PATH_INFO: `/v3/projects/d243a558d46347cbad332e46e5096940` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:46.106830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b68f8-d230-4298-a5e5-6daf708607c8 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:46.107126 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b68f8-d230-4298-a5e5-6daf708607c8 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:46.107409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b68f8-d230-4298-a5e5-6daf708607c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:46.108472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a29624-10c2-4dae-a063-fb06358f8963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d243a558d46347cbad332e46e5096940)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:46.113737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a29624-10c2-4dae-a063-fb06358f8963 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:46.152461 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-2d6b68f8-d230-4298-a5e5-6daf708607c8 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:46.153886 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2d6b68f8-d230-4298-a5e5-6daf708607c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:46.251939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:46.268767 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1091/2194] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:46.270948 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-02a29624-10c2-4dae-a063-fb06358f8963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd243a558d46347cbad332e46e5096940', 'request_id': 'req-02a29624-10c2-4dae-a063-fb06358f8963'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:46.279586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b68f8-d230-4298-a5e5-6daf708607c8 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:46.281389 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2d6b68f8-d230-4298-a5e5-6daf708607c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:46.284629 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-02a29624-10c2-4dae-a063-fb06358f8963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd243a558d46347cbad332e46e5096940', 'request_id': 'req-02a29624-10c2-4dae-a063-fb06358f8963'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:46.303627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-02a29624-10c2-4dae-a063-fb06358f8963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd243a558d46347cbad332e46e5096940', 'request_id': 'req-02a29624-10c2-4dae-a063-fb06358f8963'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:46.338822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-02a29624-10c2-4dae-a063-fb06358f8963 admin admin] The token cache is being invalidate because project d243a558d46347cbad332e46e5096940 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:46.339421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-02a29624-10c2-4dae-a063-fb06358f8963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:46.340309 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-02a29624-10c2-4dae-a063-fb06358f8963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:46.351211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1104/2195] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:46 2022] DELETE /identity/v3/projects/d243a558d46347cbad332e46e5096940 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:46.353777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a29624-10c2-4dae-a063-fb06358f8963 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:46.355996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-02a29624-10c2-4dae-a063-fb06358f8963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:46.381421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b68f8-d230-4298-a5e5-6daf708607c8 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:46.528087 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099e76a-5d2d-40bd-a66a-1da9af36991c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:46.528741 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099e76a-5d2d-40bd-a66a-1da9af36991c service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:46.529025 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099e76a-5d2d-40bd-a66a-1da9af36991c service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:46.529323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099e76a-5d2d-40bd-a66a-1da9af36991c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:46.535758 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a29624-10c2-4dae-a063-fb06358f8963 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:46.538349 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-9099e76a-5d2d-40bd-a66a-1da9af36991c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:46.550540 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374e53d-d37a-44fc-bee2-ab30b64b86b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:46.551219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e53d-d37a-44fc-bee2-ab30b64b86b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:46.551503 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e53d-d37a-44fc-bee2-ab30b64b86b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:46.552050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374e53d-d37a-44fc-bee2-ab30b64b86b2 admin admin] PATH_INFO: `/v3/users/fac0ceba35b5469489a22035b5be1839` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:46.554068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374e53d-d37a-44fc-bee2-ab30b64b86b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac0ceba35b5469489a22035b5be1839)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:46.556045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e374e53d-d37a-44fc-bee2-ab30b64b86b2 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:46.558219 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099e76a-5d2d-40bd-a66a-1da9af36991c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:46.559650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099e76a-5d2d-40bd-a66a-1da9af36991c service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:46.589691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1092/2196] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:24:46.668001 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e374e53d-d37a-44fc-bee2-ab30b64b86b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac0ceba35b5469489a22035b5be1839'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:46.669497 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e374e53d-d37a-44fc-bee2-ab30b64b86b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac0ceba35b5469489a22035b5be1839'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:46.691804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e374e53d-d37a-44fc-bee2-ab30b64b86b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac0ceba35b5469489a22035b5be1839'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:46.692522 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e374e53d-d37a-44fc-bee2-ab30b64b86b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac0ceba35b5469489a22035b5be1839'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:46.715396 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e374e53d-d37a-44fc-bee2-ab30b64b86b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac0ceba35b5469489a22035b5be1839'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:46.774015 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1105/2197] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:46 2022] DELETE /identity/v3/users/fac0ceba35b5469489a22035b5be1839 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:47.170376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099e76a-5d2d-40bd-a66a-1da9af36991c service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:47.172231 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-9099e76a-5d2d-40bd-a66a-1da9af36991c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:47.202822 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099e76a-5d2d-40bd-a66a-1da9af36991c service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:47.227234 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:47.357007 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:47.358351 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:47.361673 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:47.362769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456 admin admin] PATH_INFO: `/v3/projects/09481cbd5faa43a483e47d61c6b038c9` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:47.364180 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09481cbd5faa43a483e47d61c6b038c9)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:47.366789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:47.534557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09481cbd5faa43a483e47d61c6b038c9', 'request_id': 'req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:47.557211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09481cbd5faa43a483e47d61c6b038c9', 'request_id': 'req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:47.576714 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09481cbd5faa43a483e47d61c6b038c9', 'request_id': 'req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:47.623338 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456 admin admin] The token cache is being invalidate because project 09481cbd5faa43a483e47d61c6b038c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:47.624007 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:47.625115 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:47.635534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1093/2198] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:47 2022] DELETE /identity/v3/projects/09481cbd5faa43a483e47d61c6b038c9 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:47.636643 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374e53d-d37a-44fc-bee2-ab30b64b86b2 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:47.639160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e374e53d-d37a-44fc-bee2-ab30b64b86b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:47.681873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374e53d-d37a-44fc-bee2-ab30b64b86b2 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:47.828303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:47.875759 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:47.878838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:47.902659 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a184c2b-f90e-4afd-b457-40ed4a527d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:47.903323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a184c2b-f90e-4afd-b457-40ed4a527d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:47.903623 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a184c2b-f90e-4afd-b457-40ed4a527d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:47.904063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a184c2b-f90e-4afd-b457-40ed4a527d2f admin admin] PATH_INFO: `/v3/users/55335d5c3c424cae94310d439e87e983` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:47.912604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a184c2b-f90e-4afd-b457-40ed4a527d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=55335d5c3c424cae94310d439e87e983)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:47.919710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a184c2b-f90e-4afd-b457-40ed4a527d2f admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:48.045256 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2a184c2b-f90e-4afd-b457-40ed4a527d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55335d5c3c424cae94310d439e87e983'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:48.050282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2a184c2b-f90e-4afd-b457-40ed4a527d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55335d5c3c424cae94310d439e87e983'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:48.067398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2a184c2b-f90e-4afd-b457-40ed4a527d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55335d5c3c424cae94310d439e87e983'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:48.068206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2a184c2b-f90e-4afd-b457-40ed4a527d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55335d5c3c424cae94310d439e87e983'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:48.077813 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2debde-8a0b-4b37-9ead-ad9a3cf43456 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:48.081641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2a184c2b-f90e-4afd-b457-40ed4a527d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55335d5c3c424cae94310d439e87e983'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:48.085824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:48.087225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:48.087734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:48.088226 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e admin admin] PATH_INFO: `/v3/projects/592de5a63f11449ca52b1961cb224bf0` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:48.089853 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=592de5a63f11449ca52b1961cb224bf0)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:48.114178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:48.127083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1106/2199] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:47 2022] DELETE /identity/v3/users/55335d5c3c424cae94310d439e87e983 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:48.267200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:48.281077 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '592de5a63f11449ca52b1961cb224bf0', 'request_id': 'req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:48.321793 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '592de5a63f11449ca52b1961cb224bf0', 'request_id': 'req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:48.357242 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '592de5a63f11449ca52b1961cb224bf0', 'request_id': 'req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:48.391358 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e admin admin] The token cache is being invalidate because project 592de5a63f11449ca52b1961cb224bf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:48.391991 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:48.393945 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:48.398285 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1094/2200] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:47 2022] DELETE /identity/v3/projects/592de5a63f11449ca52b1961cb224bf0 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:48.402488 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a184c2b-f90e-4afd-b457-40ed4a527d2f admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:48.404638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2a184c2b-f90e-4afd-b457-40ed4a527d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:48.533112 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a184c2b-f90e-4afd-b457-40ed4a527d2f admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:48.644288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0d9f0-1154-4ba3-8b5d-c82ae0dafa82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:48.645923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0d9f0-1154-4ba3-8b5d-c82ae0dafa82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:48.646384 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0d9f0-1154-4ba3-8b5d-c82ae0dafa82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:48.646805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0d9f0-1154-4ba3-8b5d-c82ae0dafa82 admin admin] PATH_INFO: `/v3/users/fac090c2bc724e1096c74fb729e14f48` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:48.648164 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0d9f0-1154-4ba3-8b5d-c82ae0dafa82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac090c2bc724e1096c74fb729e14f48)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:48.654189 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0d9f0-1154-4ba3-8b5d-c82ae0dafa82 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:48.699696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:48.793042 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c3e0d9f0-1154-4ba3-8b5d-c82ae0dafa82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac090c2bc724e1096c74fb729e14f48'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:48.793850 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c3e0d9f0-1154-4ba3-8b5d-c82ae0dafa82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac090c2bc724e1096c74fb729e14f48'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:48.815308 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c3e0d9f0-1154-4ba3-8b5d-c82ae0dafa82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac090c2bc724e1096c74fb729e14f48'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:48.816313 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c3e0d9f0-1154-4ba3-8b5d-c82ae0dafa82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac090c2bc724e1096c74fb729e14f48'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:48.827902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c3e0d9f0-1154-4ba3-8b5d-c82ae0dafa82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac090c2bc724e1096c74fb729e14f48'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:48.862839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1107/2201] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:48 2022] DELETE /identity/v3/users/fac090c2bc724e1096c74fb729e14f48 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:49.229064 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:49.232893 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:49.262834 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22635b6-1bb0-4cc7-ba5d-4e8e86101b6e admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:49.452720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0d9f0-1154-4ba3-8b5d-c82ae0dafa82 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:49.454418 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-c3e0d9f0-1154-4ba3-8b5d-c82ae0dafa82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:49.512735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1f63b-3ad5-4c6e-a203-ce0264616375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:49.514070 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1f63b-3ad5-4c6e-a203-ce0264616375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:49.514516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1f63b-3ad5-4c6e-a203-ce0264616375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:49.514937 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1f63b-3ad5-4c6e-a203-ce0264616375 admin admin] PATH_INFO: `/v3/projects/b55a3d1563ff412a8dfe2ef9a9f8f522` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:49.516280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1f63b-3ad5-4c6e-a203-ce0264616375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b55a3d1563ff412a8dfe2ef9a9f8f522)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:49.519312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1f63b-3ad5-4c6e-a203-ce0264616375 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:49.640997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0d9f0-1154-4ba3-8b5d-c82ae0dafa82 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:49.651165 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d61672-cc50-453b-a7f3-442430ae1a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:49.653237 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d61672-cc50-453b-a7f3-442430ae1a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:49.653521 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d61672-cc50-453b-a7f3-442430ae1a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:49.653792 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d61672-cc50-453b-a7f3-442430ae1a4e admin admin] PATH_INFO: `/v3/projects/abc91942844f4bd2bbe13ceaf3abc071` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:49.659547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d61672-cc50-453b-a7f3-442430ae1a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=abc91942844f4bd2bbe13ceaf3abc071)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:49.661517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d61672-cc50-453b-a7f3-442430ae1a4e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:49.713396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-60a1f63b-3ad5-4c6e-a203-ce0264616375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b55a3d1563ff412a8dfe2ef9a9f8f522', 'request_id': 'req-60a1f63b-3ad5-4c6e-a203-ce0264616375'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:49.729463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-60a1f63b-3ad5-4c6e-a203-ce0264616375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b55a3d1563ff412a8dfe2ef9a9f8f522', 'request_id': 'req-60a1f63b-3ad5-4c6e-a203-ce0264616375'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:49.760320 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-60a1f63b-3ad5-4c6e-a203-ce0264616375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b55a3d1563ff412a8dfe2ef9a9f8f522', 'request_id': 'req-60a1f63b-3ad5-4c6e-a203-ce0264616375'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:49.815123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-60a1f63b-3ad5-4c6e-a203-ce0264616375 admin admin] The token cache is being invalidate because project b55a3d1563ff412a8dfe2ef9a9f8f522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:49.816097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-60a1f63b-3ad5-4c6e-a203-ce0264616375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:49.816929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-60a1f63b-3ad5-4c6e-a203-ce0264616375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:49.825744 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1095/2202] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:49 2022] DELETE /identity/v3/projects/b55a3d1563ff412a8dfe2ef9a9f8f522 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:49.837824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f754a7-f2ae-49f3-9f02-bac79a08ad4e None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:49.838153 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f754a7-f2ae-49f3-9f02-bac79a08ad4e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:49.838434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f754a7-f2ae-49f3-9f02-bac79a08ad4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:49.845912 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d3d61672-cc50-453b-a7f3-442430ae1a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abc91942844f4bd2bbe13ceaf3abc071', 'request_id': 'req-d3d61672-cc50-453b-a7f3-442430ae1a4e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:49.857632 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d3d61672-cc50-453b-a7f3-442430ae1a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abc91942844f4bd2bbe13ceaf3abc071', 'request_id': 'req-d3d61672-cc50-453b-a7f3-442430ae1a4e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:49.873291 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d3d61672-cc50-453b-a7f3-442430ae1a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abc91942844f4bd2bbe13ceaf3abc071', 'request_id': 'req-d3d61672-cc50-453b-a7f3-442430ae1a4e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:49.894153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d3d61672-cc50-453b-a7f3-442430ae1a4e admin admin] The token cache is being invalidate because project abc91942844f4bd2bbe13ceaf3abc071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:49.894459 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d3d61672-cc50-453b-a7f3-442430ae1a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:49.896531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d3d61672-cc50-453b-a7f3-442430ae1a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:49.898859 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1108/2203] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:49 2022] DELETE /identity/v3/projects/abc91942844f4bd2bbe13ceaf3abc071 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:24:49.904926 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ff6fc-747f-4c8a-bdd8-a5def490abcd None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:49.905239 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ff6fc-747f-4c8a-bdd8-a5def490abcd None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:49.905516 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ff6fc-747f-4c8a-bdd8-a5def490abcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:49.905636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-40f754a7-f2ae-49f3-9f02-bac79a08ad4e None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:49.906925 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-40f754a7-f2ae-49f3-9f02-bac79a08ad4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:49.959979 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-947ff6fc-747f-4c8a-bdd8-a5def490abcd None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:49.960455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-947ff6fc-747f-4c8a-bdd8-a5def490abcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:49.992784 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:50.046744 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1096/2204] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:50.068707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1109/2205] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:50.070601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fa7d2-3999-4d42-905f-74b59916e625 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:50.071233 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fa7d2-3999-4d42-905f-74b59916e625 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:50.071658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fa7d2-3999-4d42-905f-74b59916e625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:50.076706 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8a2ed-7384-4d31-b7f3-442f0312ce74 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:50.077060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8a2ed-7384-4d31-b7f3-442f0312ce74 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:50.077425 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8a2ed-7384-4d31-b7f3-442f0312ce74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:50.096439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:50.109895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7cf8a2ed-7384-4d31-b7f3-442f0312ce74 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:50.111126 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7cf8a2ed-7384-4d31-b7f3-442f0312ce74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:50.122105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-088fa7d2-3999-4d42-905f-74b59916e625 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:50.123820 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-088fa7d2-3999-4d42-905f-74b59916e625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:50.128207 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1110/2206] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:50.134198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8a2ed-7384-4d31-b7f3-442f0312ce74 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:50.136550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1097/2207] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:50.142412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fa7d2-3999-4d42-905f-74b59916e625 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:50.157126 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8a2ed-7384-4d31-b7f3-442f0312ce74 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:50.178109 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fa7d2-3999-4d42-905f-74b59916e625 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:50.183367 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ae6ce-7435-4a3a-aa99-3cd88f31f744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:50.184204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ae6ce-7435-4a3a-aa99-3cd88f31f744 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:50.184523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ae6ce-7435-4a3a-aa99-3cd88f31f744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:50.188659 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ae6ce-7435-4a3a-aa99-3cd88f31f744 admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:50.190469 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ae6ce-7435-4a3a-aa99-3cd88f31f744 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:24:50.193973 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573167a-c105-4f47-8ab2-46e6c0490494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:50.194796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ae6ce-7435-4a3a-aa99-3cd88f31f744 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:50.196117 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573167a-c105-4f47-8ab2-46e6c0490494 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:50.196429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573167a-c105-4f47-8ab2-46e6c0490494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:50.196708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573167a-c105-4f47-8ab2-46e6c0490494 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:50.199450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ae6ce-7435-4a3a-aa99-3cd88f31f744 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:50.201355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573167a-c105-4f47-8ab2-46e6c0490494 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:24:50.201673 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573167a-c105-4f47-8ab2-46e6c0490494 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:50.203436 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573167a-c105-4f47-8ab2-46e6c0490494 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:50.231155 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1111/2208] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:24:50.235635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ae6ce-7435-4a3a-aa99-3cd88f31f744 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:50.264151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ae6ce-7435-4a3a-aa99-3cd88f31f744 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:50.269754 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1098/2209] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:24:50.277837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573167a-c105-4f47-8ab2-46e6c0490494 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:50.286323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf882aa-8b4e-415b-b8f6-c74eda32125b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:50.287086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf882aa-8b4e-415b-b8f6-c74eda32125b admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:50.288948 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf882aa-8b4e-415b-b8f6-c74eda32125b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:50.289444 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf882aa-8b4e-415b-b8f6-c74eda32125b admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:50.290208 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf882aa-8b4e-415b-b8f6-c74eda32125b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:50.295861 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf882aa-8b4e-415b-b8f6-c74eda32125b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:50.304623 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573167a-c105-4f47-8ab2-46e6c0490494 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:50.320947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1b7ea-5999-49e9-8a98-5cb4c353bdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:50.321857 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1b7ea-5999-49e9-8a98-5cb4c353bdd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:50.322370 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1b7ea-5999-49e9-8a98-5cb4c353bdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:50.322771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1b7ea-5999-49e9-8a98-5cb4c353bdd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:50.323534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b1b7ea-5999-49e9-8a98-5cb4c353bdd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:50.333935 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b1b7ea-5999-49e9-8a98-5cb4c353bdd9 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:50.386303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1112/2210] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:50 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:50.388398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf882aa-8b4e-415b-b8f6-c74eda32125b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:50.435365 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1099/2211] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:50 2022] POST /identity/v3/projects => generated 363 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:50.439683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1b7ea-5999-49e9-8a98-5cb4c353bdd9 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:50.632681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf882aa-8b4e-415b-b8f6-c74eda32125b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:50.639055 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea13b8-f2d0-4661-8bf1-d8d21e04b94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:50.639667 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea13b8-f2d0-4661-8bf1-d8d21e04b94c admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:50.640032 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea13b8-f2d0-4661-8bf1-d8d21e04b94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:50.640205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea13b8-f2d0-4661-8bf1-d8d21e04b94c admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:50.640838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea13b8-f2d0-4661-8bf1-d8d21e04b94c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:50.642778 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea13b8-f2d0-4661-8bf1-d8d21e04b94c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:50.643240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1b7ea-5999-49e9-8a98-5cb4c353bdd9 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:50.652644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a071510-e090-40c9-8158-6d7145fd46aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:50.654034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a071510-e090-40c9-8158-6d7145fd46aa admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:50.654418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a071510-e090-40c9-8158-6d7145fd46aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:50.655154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a071510-e090-40c9-8158-6d7145fd46aa admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:50.656017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a071510-e090-40c9-8158-6d7145fd46aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:50.659857 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a071510-e090-40c9-8158-6d7145fd46aa admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:50.679035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1113/2212] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:50 2022] POST /identity/v3/users => generated 291 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:50.686752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea13b8-f2d0-4661-8bf1-d8d21e04b94c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:50.702922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea13b8-f2d0-4661-8bf1-d8d21e04b94c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:50.708136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1100/2213] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:50 2022] POST /identity/v3/users => generated 282 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:50.714790 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247e7e4-ccac-4cc0-bced-549e34e033ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:50.715303 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a071510-e090-40c9-8158-6d7145fd46aa admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:50.715474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247e7e4-ccac-4cc0-bced-549e34e033ca admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:50.716086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247e7e4-ccac-4cc0-bced-549e34e033ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:50.716426 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247e7e4-ccac-4cc0-bced-549e34e033ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:50.717198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247e7e4-ccac-4cc0-bced-549e34e033ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:50.719076 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247e7e4-ccac-4cc0-bced-549e34e033ca admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:50.736144 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a071510-e090-40c9-8158-6d7145fd46aa admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:50.746195 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788714c-33d7-4351-82da-8d2584934ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:50.746996 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788714c-33d7-4351-82da-8d2584934ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:50.747404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788714c-33d7-4351-82da-8d2584934ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:50.747779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788714c-33d7-4351-82da-8d2584934ba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:50.753630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788714c-33d7-4351-82da-8d2584934ba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:50.754798 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1114/2214] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:50.754960 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788714c-33d7-4351-82da-8d2584934ba7 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:50.756939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247e7e4-ccac-4cc0-bced-549e34e033ca admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:50.772796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247e7e4-ccac-4cc0-bced-549e34e033ca admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:50.774271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1101/2215] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:50.777719 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788714c-33d7-4351-82da-8d2584934ba7 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:50.785568 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bccdd-2b83-4118-9b96-3eda02f9e0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:50.786193 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bccdd-2b83-4118-9b96-3eda02f9e0bd admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:50.786472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bccdd-2b83-4118-9b96-3eda02f9e0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:50.786960 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676bccdd-2b83-4118-9b96-3eda02f9e0bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:50.788804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bccdd-2b83-4118-9b96-3eda02f9e0bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:50.790437 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676bccdd-2b83-4118-9b96-3eda02f9e0bd admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:50.798650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788714c-33d7-4351-82da-8d2584934ba7 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:50.813424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8012e7-6857-428b-9d2d-b7bbb18f6e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:50.814240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8012e7-6857-428b-9d2d-b7bbb18f6e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:50.814743 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8012e7-6857-428b-9d2d-b7bbb18f6e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:50.815308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8012e7-6857-428b-9d2d-b7bbb18f6e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:50.818749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1115/2216] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:50.819023 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8012e7-6857-428b-9d2d-b7bbb18f6e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:50.819349 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8012e7-6857-428b-9d2d-b7bbb18f6e4b admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:50.824816 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bccdd-2b83-4118-9b96-3eda02f9e0bd admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:50.859551 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676bccdd-2b83-4118-9b96-3eda02f9e0bd admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:50.865683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef7f17-b165-4c8b-be08-b258bc26517d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:50.866420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef7f17-b165-4c8b-be08-b258bc26517d admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:50.866809 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef7f17-b165-4c8b-be08-b258bc26517d admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:50.867192 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef7f17-b165-4c8b-be08-b258bc26517d admin admin] PATH_INFO: `/v3/projects/d293d363d001462b8996f12ecf1b3587/users/32deef7d2cc3436392866741739a2d29/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:50.875057 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1102/2217] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:24:50.876466 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8012e7-6857-428b-9d2d-b7bbb18f6e4b admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:50.902265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8012e7-6857-428b-9d2d-b7bbb18f6e4b admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:50.904637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef7f17-b165-4c8b-be08-b258bc26517d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d293d363d001462b8996f12ecf1b3587, user_id=32deef7d2cc3436392866741739a2d29, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:50.913033 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef7f17-b165-4c8b-be08-b258bc26517d admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:50.917769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec7c20-c111-4486-b9d0-67bacf4c275c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:50.918801 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec7c20-c111-4486-b9d0-67bacf4c275c admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:50.919077 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec7c20-c111-4486-b9d0-67bacf4c275c admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:50.919359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec7c20-c111-4486-b9d0-67bacf4c275c admin admin] PATH_INFO: `/v3/projects/24e65e1dbeba4e308637172bc6eb936a/users/81028b0ff7f44561981be953810eb612/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:50.961914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1116/2218] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:50 2022] PUT /identity/v3/projects/d293d363d001462b8996f12ecf1b3587/users/32deef7d2cc3436392866741739a2d29/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:50.962028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec7c20-c111-4486-b9d0-67bacf4c275c admin admin] RBAC: Authorizing `identity:create_grant(project_id=24e65e1dbeba4e308637172bc6eb936a, user_id=81028b0ff7f44561981be953810eb612, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:50.962028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec7c20-c111-4486-b9d0-67bacf4c275c admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:50.963108 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2027565-6113-4ea4-9d27-76e38b85fcc6 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:50.963533 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2027565-6113-4ea4-9d27-76e38b85fcc6 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:50.963850 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2027565-6113-4ea4-9d27-76e38b85fcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:50.995691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1103/2219] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:50 2022] PUT /identity/v3/projects/24e65e1dbeba4e308637172bc6eb936a/users/81028b0ff7f44561981be953810eb612/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:51.007921 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbaa9d-4a83-435c-8c00-8d969ace06f2 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:51.008509 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbaa9d-4a83-435c-8c00-8d969ace06f2 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:51.008509 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbaa9d-4a83-435c-8c00-8d969ace06f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:51.028422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-b2027565-6113-4ea4-9d27-76e38b85fcc6 None None] MFA Rules not processed for user `32deef7d2cc3436392866741739a2d29`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:51.034815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b2027565-6113-4ea4-9d27-76e38b85fcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:51.077835 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:51.087324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-31fbaa9d-4a83-435c-8c00-8d969ace06f2 None None] MFA Rules not processed for user `81028b0ff7f44561981be953810eb612`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:51.092102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-31fbaa9d-4a83-435c-8c00-8d969ace06f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:51.207116 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1117/2220] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:51.211200 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2027565-6113-4ea4-9d27-76e38b85fcc6 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:51.285907 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:51.322768 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1104/2221] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:51.327205 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbaa9d-4a83-435c-8c00-8d969ace06f2 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:51.348526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2027565-6113-4ea4-9d27-76e38b85fcc6 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:51.362195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6c746-e16b-4ad6-8ded-908fa9cc1c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:51.363153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6c746-e16b-4ad6-8ded-908fa9cc1c3d admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:51.363676 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6c746-e16b-4ad6-8ded-908fa9cc1c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:51.363962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6c746-e16b-4ad6-8ded-908fa9cc1c3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:51.364647 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f6c746-e16b-4ad6-8ded-908fa9cc1c3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:51.366905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f6c746-e16b-4ad6-8ded-908fa9cc1c3d admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:51.418261 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1118/2222] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:51 2022] POST /identity/v3/projects => generated 381 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:51.420353 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6c746-e16b-4ad6-8ded-908fa9cc1c3d admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:51.504331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbaa9d-4a83-435c-8c00-8d969ace06f2 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:51.520647 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6c746-e16b-4ad6-8ded-908fa9cc1c3d admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:51.531882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb4cfe-59b2-42aa-bb42-9967a4043f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:51.532509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb4cfe-59b2-42aa-bb42-9967a4043f56 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:51.532793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb4cfe-59b2-42aa-bb42-9967a4043f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:51.533087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb4cfe-59b2-42aa-bb42-9967a4043f56 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:51.536813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb4cfe-59b2-42aa-bb42-9967a4043f56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:51.539005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb4cfe-59b2-42aa-bb42-9967a4043f56 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:51.570248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1119/2223] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:51 2022] POST /identity/v3/users => generated 297 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:51.571915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb4cfe-59b2-42aa-bb42-9967a4043f56 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:51.583922 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335aa92-6d12-4067-b4c3-f34a3e9170b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:51.584762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335aa92-6d12-4067-b4c3-f34a3e9170b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:51.585113 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335aa92-6d12-4067-b4c3-f34a3e9170b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:51.585244 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb4cfe-59b2-42aa-bb42-9967a4043f56 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:51.585563 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4335aa92-6d12-4067-b4c3-f34a3e9170b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:51.586415 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335aa92-6d12-4067-b4c3-f34a3e9170b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:51.591938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a8075-5506-4c60-8802-94927aee9c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:51.592595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a8075-5506-4c60-8802-94927aee9c34 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:51.592876 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a8075-5506-4c60-8802-94927aee9c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:51.593885 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a8075-5506-4c60-8802-94927aee9c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:51.594501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a8075-5506-4c60-8802-94927aee9c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:51.597849 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a8075-5506-4c60-8802-94927aee9c34 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:51.600702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4335aa92-6d12-4067-b4c3-f34a3e9170b6 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:51.620274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1120/2224] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:24:51.624502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a8075-5506-4c60-8802-94927aee9c34 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:51.644288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a8075-5506-4c60-8802-94927aee9c34 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:51.652761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ad017-2158-4316-93b2-fbb7469da466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:51.653687 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ad017-2158-4316-93b2-fbb7469da466 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:51.654067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ad017-2158-4316-93b2-fbb7469da466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:51.654596 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ad017-2158-4316-93b2-fbb7469da466 admin admin] PATH_INFO: `/v3/projects/71d93d8c8e70494082870a328dc43dfa/users/6394dd317e5a4d3faf221de182b9a565/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:51.666420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1105/2225] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:51 2022] POST /identity/v3/projects => generated 365 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:51.670747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335aa92-6d12-4067-b4c3-f34a3e9170b6 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:51.692591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ad017-2158-4316-93b2-fbb7469da466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71d93d8c8e70494082870a328dc43dfa, user_id=6394dd317e5a4d3faf221de182b9a565, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:51.700624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ad017-2158-4316-93b2-fbb7469da466 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:51.703198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:51.735346 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1121/2226] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:51 2022] PUT /identity/v3/projects/71d93d8c8e70494082870a328dc43dfa/users/6394dd317e5a4d3faf221de182b9a565/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:24:51.741916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ad017-2158-4316-93b2-fbb7469da466 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:51.862921 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ad017-2158-4316-93b2-fbb7469da466 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:51.867798 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4335aa92-6d12-4067-b4c3-f34a3e9170b6 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:51.868843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903434e-7ebb-4930-9836-b096b7c9dfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:51.870070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903434e-7ebb-4930-9836-b096b7c9dfae admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:51.870502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903434e-7ebb-4930-9836-b096b7c9dfae admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:51.870928 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903434e-7ebb-4930-9836-b096b7c9dfae admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:51.872053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7903434e-7ebb-4930-9836-b096b7c9dfae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:51.874625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7903434e-7ebb-4930-9836-b096b7c9dfae admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:51.878567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0cbe3-75c7-4227-b7ca-c352ad302d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:51.879253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0cbe3-75c7-4227-b7ca-c352ad302d90 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:51.879741 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0cbe3-75c7-4227-b7ca-c352ad302d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:51.894228 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0cbe3-75c7-4227-b7ca-c352ad302d90 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:51.896984 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0cbe3-75c7-4227-b7ca-c352ad302d90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:51.900694 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1122/2227] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:51.905413 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903434e-7ebb-4930-9836-b096b7c9dfae admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:51.905875 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0cbe3-75c7-4227-b7ca-c352ad302d90 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:51.936584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903434e-7ebb-4930-9836-b096b7c9dfae admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:51.950374 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb8b22-b43c-49be-aab9-287aa6cf3eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:51.952028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb8b22-b43c-49be-aab9-287aa6cf3eac admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:51.952418 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb8b22-b43c-49be-aab9-287aa6cf3eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:51.952724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb8b22-b43c-49be-aab9-287aa6cf3eac admin admin] PATH_INFO: `/v3/projects/71d93d8c8e70494082870a328dc43dfa/users/6394dd317e5a4d3faf221de182b9a565/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:51.961074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb8b22-b43c-49be-aab9-287aa6cf3eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=71d93d8c8e70494082870a328dc43dfa, user_id=6394dd317e5a4d3faf221de182b9a565, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:51.967180 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1106/2228] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:51 2022] POST /identity/v3/users => generated 289 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:51.968459 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb8b22-b43c-49be-aab9-287aa6cf3eac admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:51.972719 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0cbe3-75c7-4227-b7ca-c352ad302d90 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:51.995434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0cbe3-75c7-4227-b7ca-c352ad302d90 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:51.996633 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1123/2229] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:51 2022] PUT /identity/v3/projects/71d93d8c8e70494082870a328dc43dfa/users/6394dd317e5a4d3faf221de182b9a565/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:52.012971 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83713d41-72d5-43d3-a471-d0825c05787f None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:52.015936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83713d41-72d5-43d3-a471-d0825c05787f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:52.017627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83713d41-72d5-43d3-a471-d0825c05787f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:52.073951 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:52.096032 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-83713d41-72d5-43d3-a471-d0825c05787f None None] MFA Rules not processed for user `6394dd317e5a4d3faf221de182b9a565`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:52.098167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-83713d41-72d5-43d3-a471-d0825c05787f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:52.199169 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56182d75-c6b7-49f5-a910-53a79a40ddc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:52.199804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56182d75-c6b7-49f5-a910-53a79a40ddc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:52.200087 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56182d75-c6b7-49f5-a910-53a79a40ddc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:52.200609 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56182d75-c6b7-49f5-a910-53a79a40ddc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:52.201177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56182d75-c6b7-49f5-a910-53a79a40ddc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:52.202763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56182d75-c6b7-49f5-a910-53a79a40ddc9 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:52.218405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1107/2230] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:52.218405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899eeb5d-5ee8-4e72-9b0f-21fa4fc17f78 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:52.218405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899eeb5d-5ee8-4e72-9b0f-21fa4fc17f78 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:52.218405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899eeb5d-5ee8-4e72-9b0f-21fa4fc17f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:52.243792 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-899eeb5d-5ee8-4e72-9b0f-21fa4fc17f78 None None] MFA Rules not processed for user `cc96808ff3fb41f085f09aa943c3d016`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:52.245489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-899eeb5d-5ee8-4e72-9b0f-21fa4fc17f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:52.260384 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1124/2231] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:52.261351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83713d41-72d5-43d3-a471-d0825c05787f None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:52.276923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83713d41-72d5-43d3-a471-d0825c05787f None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:52.283285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016bcff5-e20c-4564-bbf8-24717ca1b473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:52.284282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bcff5-e20c-4564-bbf8-24717ca1b473 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:52.284681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bcff5-e20c-4564-bbf8-24717ca1b473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:52.284962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bcff5-e20c-4564-bbf8-24717ca1b473 admin admin] PATH_INFO: `/v3/projects/a726cf9fcef74ff48ca9569b2e8d3023/users/0b0facee181c4109a33c701132ebe946/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:52.308213 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016bcff5-e20c-4564-bbf8-24717ca1b473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a726cf9fcef74ff48ca9569b2e8d3023, user_id=0b0facee181c4109a33c701132ebe946, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:52.311521 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016bcff5-e20c-4564-bbf8-24717ca1b473 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:52.333845 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1125/2232] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:52 2022] PUT /identity/v3/projects/a726cf9fcef74ff48ca9569b2e8d3023/users/0b0facee181c4109a33c701132ebe946/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:52.335246 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016bcff5-e20c-4564-bbf8-24717ca1b473 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:52.338684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-016bcff5-e20c-4564-bbf8-24717ca1b473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:52.342826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1108/2233] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:52.343940 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899eeb5d-5ee8-4e72-9b0f-21fa4fc17f78 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:52.381397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899eeb5d-5ee8-4e72-9b0f-21fa4fc17f78 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:52.424982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:52.460243 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016bcff5-e20c-4564-bbf8-24717ca1b473 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:52.467770 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e23cf-5b79-4d2b-8465-31532a7f2ae4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:52.469241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e23cf-5b79-4d2b-8465-31532a7f2ae4 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:52.469565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e23cf-5b79-4d2b-8465-31532a7f2ae4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:52.470442 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e23cf-5b79-4d2b-8465-31532a7f2ae4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:52.525351 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71131857-8606-483b-9bf7-30c34860c008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:52.526028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71131857-8606-483b-9bf7-30c34860c008 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:52.526312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71131857-8606-483b-9bf7-30c34860c008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:52.526591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71131857-8606-483b-9bf7-30c34860c008 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:52.541099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71131857-8606-483b-9bf7-30c34860c008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:52.542849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71131857-8606-483b-9bf7-30c34860c008 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:52.579021 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1109/2234] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:52.583654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71131857-8606-483b-9bf7-30c34860c008 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:52.721894 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71131857-8606-483b-9bf7-30c34860c008 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:52.742296 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e23cf-5b79-4d2b-8465-31532a7f2ae4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:52.743788 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e23cf-5b79-4d2b-8465-31532a7f2ae4 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:52.840235 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1126/2235] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:52.843769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3938c7-9743-4eb6-b3c8-0da9b78571a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:52.844465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3938c7-9743-4eb6-b3c8-0da9b78571a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:52.848633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3938c7-9743-4eb6-b3c8-0da9b78571a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:52.850014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3938c7-9743-4eb6-b3c8-0da9b78571a4 admin admin] PATH_INFO: `/v3/projects/a726cf9fcef74ff48ca9569b2e8d3023/users/0b0facee181c4109a33c701132ebe946/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:52.855472 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3938c7-9743-4eb6-b3c8-0da9b78571a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a726cf9fcef74ff48ca9569b2e8d3023, user_id=0b0facee181c4109a33c701132ebe946, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:52.860809 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3938c7-9743-4eb6-b3c8-0da9b78571a4 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:52.916001 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1110/2236] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:52 2022] PUT /identity/v3/projects/a726cf9fcef74ff48ca9569b2e8d3023/users/0b0facee181c4109a33c701132ebe946/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:52.919273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6f15d-725b-44cd-86c1-b08ada2b3688 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:52.919646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6f15d-725b-44cd-86c1-b08ada2b3688 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:52.920291 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6f15d-725b-44cd-86c1-b08ada2b3688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:53.010119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3938c7-9743-4eb6-b3c8-0da9b78571a4 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:53.014248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-d3b6f15d-725b-44cd-86c1-b08ada2b3688 None None] MFA Rules not processed for user `0b0facee181c4109a33c701132ebe946`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:53.015583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d3b6f15d-725b-44cd-86c1-b08ada2b3688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:53.104679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:53.151948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3938c7-9743-4eb6-b3c8-0da9b78571a4 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:53.161913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb8894-c9e4-44b8-9c97-90c8f816a422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:53.162784 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb8894-c9e4-44b8-9c97-90c8f816a422 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:53.164772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb8894-c9e4-44b8-9c97-90c8f816a422 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:53.173299 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb8894-c9e4-44b8-9c97-90c8f816a422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:53.221814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1127/2237] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:53.303964 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb8894-c9e4-44b8-9c97-90c8f816a422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:53.305871 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb8894-c9e4-44b8-9c97-90c8f816a422 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:53.331892 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1111/2238] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:53.417351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6f15d-725b-44cd-86c1-b08ada2b3688 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:53.432850 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6f15d-725b-44cd-86c1-b08ada2b3688 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:53.437280 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa6848-a5a8-4f39-ae69-0dc6c76feaef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:53.438189 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa6848-a5a8-4f39-ae69-0dc6c76feaef service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:53.438637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa6848-a5a8-4f39-ae69-0dc6c76feaef service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:53.439104 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa6848-a5a8-4f39-ae69-0dc6c76feaef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:53.449844 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa6848-a5a8-4f39-ae69-0dc6c76feaef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:53.451609 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa6848-a5a8-4f39-ae69-0dc6c76feaef service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:53.477388 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1128/2239] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:53 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:53.672455 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb8894-c9e4-44b8-9c97-90c8f816a422 service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:53.693736 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb8894-c9e4-44b8-9c97-90c8f816a422 service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:53.698943 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b80fe2-ff52-48b9-88e0-3b46b0d60454 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:53.702735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b80fe2-ff52-48b9-88e0-3b46b0d60454 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:53.703158 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b80fe2-ff52-48b9-88e0-3b46b0d60454 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:53.703445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b80fe2-ff52-48b9-88e0-3b46b0d60454 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:53.884677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b80fe2-ff52-48b9-88e0-3b46b0d60454 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:53.888302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b80fe2-ff52-48b9-88e0-3b46b0d60454 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:53.937343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1112/2240] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:54.461912 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa6848-a5a8-4f39-ae69-0dc6c76feaef service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:54.626682 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa6848-a5a8-4f39-ae69-0dc6c76feaef service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:54.660283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1171031-4f35-4832-9514-44c392a82e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:54.661923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1171031-4f35-4832-9514-44c392a82e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:54.662799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1171031-4f35-4832-9514-44c392a82e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:54.664517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1171031-4f35-4832-9514-44c392a82e5f admin admin] PATH_INFO: `/v3/users/32deef7d2cc3436392866741739a2d29` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:54.667877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1171031-4f35-4832-9514-44c392a82e5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=32deef7d2cc3436392866741739a2d29)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:54.670286 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1171031-4f35-4832-9514-44c392a82e5f admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:54.676368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:54.789286 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c1171031-4f35-4832-9514-44c392a82e5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32deef7d2cc3436392866741739a2d29'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:54.790775 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c1171031-4f35-4832-9514-44c392a82e5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32deef7d2cc3436392866741739a2d29'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:54.811506 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c1171031-4f35-4832-9514-44c392a82e5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32deef7d2cc3436392866741739a2d29'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:54.812232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c1171031-4f35-4832-9514-44c392a82e5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32deef7d2cc3436392866741739a2d29'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:54.821882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c1171031-4f35-4832-9514-44c392a82e5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32deef7d2cc3436392866741739a2d29'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:54.879600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1129/2241] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:54 2022] DELETE /identity/v3/users/32deef7d2cc3436392866741739a2d29 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:54.880662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e6455-c60f-4eea-9dd3-0f54973696a8 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:54.881160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e6455-c60f-4eea-9dd3-0f54973696a8 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:54.881596 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e6455-c60f-4eea-9dd3-0f54973696a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:54.918176 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-303e6455-c60f-4eea-9dd3-0f54973696a8 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:54.920813 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-303e6455-c60f-4eea-9dd3-0f54973696a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:55.022655 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1113/2242] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:55.036160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1171031-4f35-4832-9514-44c392a82e5f admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:55.038051 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-c1171031-4f35-4832-9514-44c392a82e5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:55.161955 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1171031-4f35-4832-9514-44c392a82e5f admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:55.168517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967a054-1b92-4105-8386-031f0d97a3ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:55.169392 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967a054-1b92-4105-8386-031f0d97a3ec service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:55.169673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967a054-1b92-4105-8386-031f0d97a3ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:55.169944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967a054-1b92-4105-8386-031f0d97a3ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:55.190501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8967a054-1b92-4105-8386-031f0d97a3ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:55.192238 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8967a054-1b92-4105-8386-031f0d97a3ec service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:55.226278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1130/2243] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:55.866123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e6455-c60f-4eea-9dd3-0f54973696a8 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:55.876631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-303e6455-c60f-4eea-9dd3-0f54973696a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:55.916950 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e6455-c60f-4eea-9dd3-0f54973696a8 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:55.939222 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3febb-5d39-48f9-b533-92c8c3741616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:55.939987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3febb-5d39-48f9-b533-92c8c3741616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:55.940468 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3febb-5d39-48f9-b533-92c8c3741616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:55.940909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3febb-5d39-48f9-b533-92c8c3741616 admin admin] PATH_INFO: `/v3/projects/d293d363d001462b8996f12ecf1b3587` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:55.942749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3febb-5d39-48f9-b533-92c8c3741616 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d293d363d001462b8996f12ecf1b3587)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:55.944833 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3febb-5d39-48f9-b533-92c8c3741616 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:56.025043 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:56.054920 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-81a3febb-5d39-48f9-b533-92c8c3741616 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd293d363d001462b8996f12ecf1b3587', 'request_id': 'req-81a3febb-5d39-48f9-b533-92c8c3741616'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:56.083844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-81a3febb-5d39-48f9-b533-92c8c3741616 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd293d363d001462b8996f12ecf1b3587', 'request_id': 'req-81a3febb-5d39-48f9-b533-92c8c3741616'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:56.098235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-81a3febb-5d39-48f9-b533-92c8c3741616 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd293d363d001462b8996f12ecf1b3587', 'request_id': 'req-81a3febb-5d39-48f9-b533-92c8c3741616'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:56.117386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-81a3febb-5d39-48f9-b533-92c8c3741616 admin admin] The token cache is being invalidate because project d293d363d001462b8996f12ecf1b3587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:56.117386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-81a3febb-5d39-48f9-b533-92c8c3741616 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:56.119126 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-81a3febb-5d39-48f9-b533-92c8c3741616 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:56.125809 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967a054-1b92-4105-8386-031f0d97a3ec service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:56.126407 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1114/2244] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:55 2022] DELETE /identity/v3/projects/d293d363d001462b8996f12ecf1b3587 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:56.139821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967a054-1b92-4105-8386-031f0d97a3ec service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:56.245083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601faf9-04ab-41d1-b30c-8dc8c88158e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:56.246011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601faf9-04ab-41d1-b30c-8dc8c88158e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:56.246596 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601faf9-04ab-41d1-b30c-8dc8c88158e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:56.246793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601faf9-04ab-41d1-b30c-8dc8c88158e9 admin admin] PATH_INFO: `/v3/users/6394dd317e5a4d3faf221de182b9a565` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:56.248373 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601faf9-04ab-41d1-b30c-8dc8c88158e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6394dd317e5a4d3faf221de182b9a565)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:56.262321 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601faf9-04ab-41d1-b30c-8dc8c88158e9 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:56.315182 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:56.360696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2601faf9-04ab-41d1-b30c-8dc8c88158e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6394dd317e5a4d3faf221de182b9a565'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:56.362448 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2601faf9-04ab-41d1-b30c-8dc8c88158e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6394dd317e5a4d3faf221de182b9a565'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:56.375462 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2601faf9-04ab-41d1-b30c-8dc8c88158e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6394dd317e5a4d3faf221de182b9a565'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:56.376570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2601faf9-04ab-41d1-b30c-8dc8c88158e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6394dd317e5a4d3faf221de182b9a565'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:56.387374 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2601faf9-04ab-41d1-b30c-8dc8c88158e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6394dd317e5a4d3faf221de182b9a565'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:56.424828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1131/2245] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:56 2022] DELETE /identity/v3/users/6394dd317e5a4d3faf221de182b9a565 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:57.144596 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3febb-5d39-48f9-b533-92c8c3741616 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:57.146707 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-81a3febb-5d39-48f9-b533-92c8c3741616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:57.169918 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3febb-5d39-48f9-b533-92c8c3741616 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:57.320499 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b5c44-7365-461e-ae40-d981d9a5a60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:57.328518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b5c44-7365-461e-ae40-d981d9a5a60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:57.328882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b5c44-7365-461e-ae40-d981d9a5a60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:57.329292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b5c44-7365-461e-ae40-d981d9a5a60b admin admin] PATH_INFO: `/v3/projects/71d93d8c8e70494082870a328dc43dfa` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:57.337564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b5c44-7365-461e-ae40-d981d9a5a60b admin admin] RBAC: Authorizing `identity:delete_project(project_id=71d93d8c8e70494082870a328dc43dfa)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:57.340036 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b5c44-7365-461e-ae40-d981d9a5a60b admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:57.460520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6f0b5c44-7365-461e-ae40-d981d9a5a60b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71d93d8c8e70494082870a328dc43dfa', 'request_id': 'req-6f0b5c44-7365-461e-ae40-d981d9a5a60b'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:57.469412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6f0b5c44-7365-461e-ae40-d981d9a5a60b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71d93d8c8e70494082870a328dc43dfa', 'request_id': 'req-6f0b5c44-7365-461e-ae40-d981d9a5a60b'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:57.480982 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6f0b5c44-7365-461e-ae40-d981d9a5a60b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71d93d8c8e70494082870a328dc43dfa', 'request_id': 'req-6f0b5c44-7365-461e-ae40-d981d9a5a60b'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:57.501465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6f0b5c44-7365-461e-ae40-d981d9a5a60b admin admin] The token cache is being invalidate because project 71d93d8c8e70494082870a328dc43dfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:57.501806 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6f0b5c44-7365-461e-ae40-d981d9a5a60b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:57.502997 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6f0b5c44-7365-461e-ae40-d981d9a5a60b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:57.508847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1115/2246] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:57 2022] DELETE /identity/v3/projects/71d93d8c8e70494082870a328dc43dfa => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:57.510580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06f76e-51c6-46e5-a9e8-afba3c5fd5ef None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:57.511484 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06f76e-51c6-46e5-a9e8-afba3c5fd5ef None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:57.512066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06f76e-51c6-46e5-a9e8-afba3c5fd5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:57.546948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b5c44-7365-461e-ae40-d981d9a5a60b admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:57.547681 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-9d06f76e-51c6-46e5-a9e8-afba3c5fd5ef None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:57.550235 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9d06f76e-51c6-46e5-a9e8-afba3c5fd5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:57.552507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-6f0b5c44-7365-461e-ae40-d981d9a5a60b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:57.663294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:57.690023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1132/2247] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:57.693088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c6eb1-1b60-4d90-8488-ed3f652e1094 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:57.693410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c6eb1-1b60-4d90-8488-ed3f652e1094 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:57.693687 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c6eb1-1b60-4d90-8488-ed3f652e1094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:57.710939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b5c44-7365-461e-ae40-d981d9a5a60b admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:57.716784 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3bb0a-ec2e-461a-ba98-6bcd75024b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:57.717725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3bb0a-ec2e-461a-ba98-6bcd75024b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:57.718115 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3bb0a-ec2e-461a-ba98-6bcd75024b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:57.718493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3bb0a-ec2e-461a-ba98-6bcd75024b5e admin admin] PATH_INFO: `/v3/users/81028b0ff7f44561981be953810eb612` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:57.719778 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3bb0a-ec2e-461a-ba98-6bcd75024b5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=81028b0ff7f44561981be953810eb612)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:57.722183 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3bb0a-ec2e-461a-ba98-6bcd75024b5e admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:57.731994 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-de3c6eb1-1b60-4d90-8488-ed3f652e1094 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:57.734801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-de3c6eb1-1b60-4d90-8488-ed3f652e1094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:57.761193 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1133/2248] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:57.763815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c6eb1-1b60-4d90-8488-ed3f652e1094 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:57.789636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c6eb1-1b60-4d90-8488-ed3f652e1094 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:57.802626 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2502cfd-3bf0-46bc-b30b-81f74dc9b077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:57.804572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2502cfd-3bf0-46bc-b30b-81f74dc9b077 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:57.804885 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2502cfd-3bf0-46bc-b30b-81f74dc9b077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:57.806258 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2502cfd-3bf0-46bc-b30b-81f74dc9b077 admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:57.807080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2502cfd-3bf0-46bc-b30b-81f74dc9b077 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:24:57.807457 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2502cfd-3bf0-46bc-b30b-81f74dc9b077 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:57.811504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2502cfd-3bf0-46bc-b30b-81f74dc9b077 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:57.847766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1134/2249] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:24:57.848847 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2502cfd-3bf0-46bc-b30b-81f74dc9b077 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:57.853053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-87d3bb0a-ec2e-461a-ba98-6bcd75024b5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81028b0ff7f44561981be953810eb612'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:57.853829 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-87d3bb0a-ec2e-461a-ba98-6bcd75024b5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81028b0ff7f44561981be953810eb612'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:57.865704 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2502cfd-3bf0-46bc-b30b-81f74dc9b077 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:57.876460 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509761e-cc9c-4bf2-b82a-0fd9723e9114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:57.877100 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509761e-cc9c-4bf2-b82a-0fd9723e9114 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:57.878066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509761e-cc9c-4bf2-b82a-0fd9723e9114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:57.878593 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509761e-cc9c-4bf2-b82a-0fd9723e9114 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:57.879306 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509761e-cc9c-4bf2-b82a-0fd9723e9114 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:57.884573 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-87d3bb0a-ec2e-461a-ba98-6bcd75024b5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81028b0ff7f44561981be953810eb612'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:57.886132 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509761e-cc9c-4bf2-b82a-0fd9723e9114 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:57.900140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-87d3bb0a-ec2e-461a-ba98-6bcd75024b5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81028b0ff7f44561981be953810eb612'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:57.901059 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-87d3bb0a-ec2e-461a-ba98-6bcd75024b5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81028b0ff7f44561981be953810eb612'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:57.921307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1116/2250] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:57 2022] DELETE /identity/v3/users/81028b0ff7f44561981be953810eb612 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:24:57.926312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35df70b-dc64-4ae7-a1d9-1663e88423da None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:57.927079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35df70b-dc64-4ae7-a1d9-1663e88423da None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:57.927202 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35df70b-dc64-4ae7-a1d9-1663e88423da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:57.946191 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1135/2251] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:24:57 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:24:57.950603 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509761e-cc9c-4bf2-b82a-0fd9723e9114 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:57.953954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5509761e-cc9c-4bf2-b82a-0fd9723e9114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:57.954467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-f35df70b-dc64-4ae7-a1d9-1663e88423da None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:57.956778 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f35df70b-dc64-4ae7-a1d9-1663e88423da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:58.084026 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1117/2252] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:58.104905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509761e-cc9c-4bf2-b82a-0fd9723e9114 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:58.116130 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681a392-25c2-47c4-a9d5-605636c64e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:58.117165 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681a392-25c2-47c4-a9d5-605636c64e8f admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:58.117284 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681a392-25c2-47c4-a9d5-605636c64e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:58.117473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681a392-25c2-47c4-a9d5-605636c64e8f admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:58.118149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681a392-25c2-47c4-a9d5-605636c64e8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:58.120067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681a392-25c2-47c4-a9d5-605636c64e8f admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:58.143652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:58.172014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1136/2253] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:24:57 2022] POST /identity/v3/users => generated 287 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:24:58.173436 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35df70b-dc64-4ae7-a1d9-1663e88423da None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:58.180215 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681a392-25c2-47c4-a9d5-605636c64e8f admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:58.181932 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-0681a392-25c2-47c4-a9d5-605636c64e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:58.206343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35df70b-dc64-4ae7-a1d9-1663e88423da None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:58.232296 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65bd7e0-3a10-460a-b43b-1e6084fe88c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:58.236605 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65bd7e0-3a10-460a-b43b-1e6084fe88c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:58.236948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65bd7e0-3a10-460a-b43b-1e6084fe88c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:58.237262 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65bd7e0-3a10-460a-b43b-1e6084fe88c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:58.237932 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65bd7e0-3a10-460a-b43b-1e6084fe88c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:58.239559 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65bd7e0-3a10-460a-b43b-1e6084fe88c8 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:58.289150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1118/2254] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:58.292928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65bd7e0-3a10-460a-b43b-1e6084fe88c8 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:58.308290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681a392-25c2-47c4-a9d5-605636c64e8f admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:58.320196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545f2a1-8046-437f-8b75-0eaed8f55e8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:58.320840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545f2a1-8046-437f-8b75-0eaed8f55e8a service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:58.321125 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545f2a1-8046-437f-8b75-0eaed8f55e8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:58.321415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545f2a1-8046-437f-8b75-0eaed8f55e8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:58.323086 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65bd7e0-3a10-460a-b43b-1e6084fe88c8 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:58.338656 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642b0d3-f883-42ad-9d84-5a291b4a26cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:58.339309 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642b0d3-f883-42ad-9d84-5a291b4a26cb admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:58.339583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642b0d3-f883-42ad-9d84-5a291b4a26cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:58.339856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642b0d3-f883-42ad-9d84-5a291b4a26cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:58.340517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642b0d3-f883-42ad-9d84-5a291b4a26cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:58.342170 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642b0d3-f883-42ad-9d84-5a291b4a26cb admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:58.348306 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545f2a1-8046-437f-8b75-0eaed8f55e8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:58.349886 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545f2a1-8046-437f-8b75-0eaed8f55e8a service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:58.375481 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1119/2255] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:24:58.376473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642b0d3-f883-42ad-9d84-5a291b4a26cb admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:58.391377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1137/2256] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:58.397567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642b0d3-f883-42ad-9d84-5a291b4a26cb admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:58.404658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94bb7b-cfed-491a-91e3-293bd083bc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:58.405699 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94bb7b-cfed-491a-91e3-293bd083bc27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:58.406295 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94bb7b-cfed-491a-91e3-293bd083bc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:58.406795 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94bb7b-cfed-491a-91e3-293bd083bc27 admin admin] PATH_INFO: `/v3/projects/cae607c773ff4e2aaadf9fcc5160810f/users/fd1669e8851b4722a747492dc610432d/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:58.459438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94bb7b-cfed-491a-91e3-293bd083bc27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cae607c773ff4e2aaadf9fcc5160810f, user_id=fd1669e8851b4722a747492dc610432d, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:58.462968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94bb7b-cfed-491a-91e3-293bd083bc27 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:58.508292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1120/2257] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:24:58 2022] PUT /identity/v3/projects/cae607c773ff4e2aaadf9fcc5160810f/users/fd1669e8851b4722a747492dc610432d/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:24:58.509333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2d601-7afb-4728-8239-80828ff8e749 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:58.509910 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2d601-7afb-4728-8239-80828ff8e749 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:58.512168 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2d601-7afb-4728-8239-80828ff8e749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:58.644590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-17c2d601-7afb-4728-8239-80828ff8e749 None None] MFA Rules not processed for user `fd1669e8851b4722a747492dc610432d`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:58.646483 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-17c2d601-7afb-4728-8239-80828ff8e749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:58.851815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1138/2258] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:24:58.864097 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94bb7b-cfed-491a-91e3-293bd083bc27 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:58.866204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5d94bb7b-cfed-491a-91e3-293bd083bc27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:58.923995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:58.986020 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94bb7b-cfed-491a-91e3-293bd083bc27 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:58.990604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4d9b0-d4e7-4aba-901a-05b82baad1aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:58.991506 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d9b0-d4e7-4aba-901a-05b82baad1aa service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:58.991883 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d9b0-d4e7-4aba-901a-05b82baad1aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:58.992531 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4d9b0-d4e7-4aba-901a-05b82baad1aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:59.003670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4d9b0-d4e7-4aba-901a-05b82baad1aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:59.004808 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4d9b0-d4e7-4aba-901a-05b82baad1aa service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:59.027804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1121/2259] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:24:59.042653 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2d601-7afb-4728-8239-80828ff8e749 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:59.124934 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2d601-7afb-4728-8239-80828ff8e749 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:59.132317 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e045140-47d2-4333-8b77-562987f16396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:59.133094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e045140-47d2-4333-8b77-562987f16396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:59.133540 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e045140-47d2-4333-8b77-562987f16396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:59.134920 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e045140-47d2-4333-8b77-562987f16396 admin admin] PATH_INFO: `/v3/users/fd1669e8851b4722a747492dc610432d` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:59.139957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e045140-47d2-4333-8b77-562987f16396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd1669e8851b4722a747492dc610432d)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:59.148421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e045140-47d2-4333-8b77-562987f16396 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:59.198557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4d9b0-d4e7-4aba-901a-05b82baad1aa service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:59.200368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b9c4d9b0-d4e7-4aba-901a-05b82baad1aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:59.211976 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:59.234490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4d9b0-d4e7-4aba-901a-05b82baad1aa service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:59.249884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c84d3-f29a-4351-ad48-c5f2971a561c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:59.251738 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c84d3-f29a-4351-ad48-c5f2971a561c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:59.252500 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c84d3-f29a-4351-ad48-c5f2971a561c admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:59.254942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c84d3-f29a-4351-ad48-c5f2971a561c admin admin] PATH_INFO: `/v3/projects/24e65e1dbeba4e308637172bc6eb936a` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:59.257567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c84d3-f29a-4351-ad48-c5f2971a561c admin admin] RBAC: Authorizing `identity:delete_project(project_id=24e65e1dbeba4e308637172bc6eb936a)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:59.261873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c84d3-f29a-4351-ad48-c5f2971a561c admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:59.273780 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-1e045140-47d2-4333-8b77-562987f16396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd1669e8851b4722a747492dc610432d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:59.275274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-1e045140-47d2-4333-8b77-562987f16396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd1669e8851b4722a747492dc610432d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:59.290981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-1e045140-47d2-4333-8b77-562987f16396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd1669e8851b4722a747492dc610432d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:59.292177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-1e045140-47d2-4333-8b77-562987f16396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd1669e8851b4722a747492dc610432d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:59.306315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-1e045140-47d2-4333-8b77-562987f16396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd1669e8851b4722a747492dc610432d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:59.345801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1139/2260] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:59 2022] DELETE /identity/v3/users/fd1669e8851b4722a747492dc610432d => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:24:59.350370 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944af81-d761-4098-af55-7f1ade1b100a None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:59.351198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944af81-d761-4098-af55-7f1ade1b100a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:59.351198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944af81-d761-4098-af55-7f1ade1b100a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:59.378900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-4944af81-d761-4098-af55-7f1ade1b100a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:24:59.380637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-4944af81-d761-4098-af55-7f1ade1b100a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:59.404372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-2b2c84d3-f29a-4351-ad48-c5f2971a561c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24e65e1dbeba4e308637172bc6eb936a', 'request_id': 'req-2b2c84d3-f29a-4351-ad48-c5f2971a561c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:59.411583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-2b2c84d3-f29a-4351-ad48-c5f2971a561c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24e65e1dbeba4e308637172bc6eb936a', 'request_id': 'req-2b2c84d3-f29a-4351-ad48-c5f2971a561c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:59.420652 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-2b2c84d3-f29a-4351-ad48-c5f2971a561c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24e65e1dbeba4e308637172bc6eb936a', 'request_id': 'req-2b2c84d3-f29a-4351-ad48-c5f2971a561c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:59.433187 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-2b2c84d3-f29a-4351-ad48-c5f2971a561c admin admin] The token cache is being invalidate because project 24e65e1dbeba4e308637172bc6eb936a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:24:59.433489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-2b2c84d3-f29a-4351-ad48-c5f2971a561c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:59.434154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-2b2c84d3-f29a-4351-ad48-c5f2971a561c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:59.436309 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1122/2261] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:24:59 2022] DELETE /identity/v3/projects/24e65e1dbeba4e308637172bc6eb936a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:24:59.440273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c84d3-f29a-4351-ad48-c5f2971a561c admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:59.442051 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2b2c84d3-f29a-4351-ad48-c5f2971a561c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:59.469220 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1140/2262] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:24:59.480937 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944af81-d761-4098-af55-7f1ade1b100a None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:24:59.482776 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-4944af81-d761-4098-af55-7f1ade1b100a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:24:59.528064 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c84d3-f29a-4351-ad48-c5f2971a561c admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:59.534746 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce93377-75a2-4a19-a821-053b7c547446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:59.535602 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce93377-75a2-4a19-a821-053b7c547446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:59.535983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce93377-75a2-4a19-a821-053b7c547446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:59.541654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce93377-75a2-4a19-a821-053b7c547446 admin admin] PATH_INFO: `/v3/users/0b0facee181c4109a33c701132ebe946` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:59.545291 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce93377-75a2-4a19-a821-053b7c547446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b0facee181c4109a33c701132ebe946)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:59.552338 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce93377-75a2-4a19-a821-053b7c547446 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:59.582365 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944af81-d761-4098-af55-7f1ade1b100a None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:24:59.588312 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d3750c-5fe1-4eff-b666-b72f63f81469 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:24:59.588944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3750c-5fe1-4eff-b666-b72f63f81469 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:24:59.589266 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3750c-5fe1-4eff-b666-b72f63f81469 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:24:59.589551 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d3750c-5fe1-4eff-b666-b72f63f81469 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:24:59.604807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d3750c-5fe1-4eff-b666-b72f63f81469 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:24:59.606453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d3750c-5fe1-4eff-b666-b72f63f81469 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:24:59.617933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1141/2263] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:24:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:24:59.620141 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-dce93377-75a2-4a19-a821-053b7c547446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b0facee181c4109a33c701132ebe946'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:59.621119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-dce93377-75a2-4a19-a821-053b7c547446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b0facee181c4109a33c701132ebe946'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:59.622384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:24:59.640166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-dce93377-75a2-4a19-a821-053b7c547446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b0facee181c4109a33c701132ebe946'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:59.656366 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-dce93377-75a2-4a19-a821-053b7c547446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b0facee181c4109a33c701132ebe946'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:59.657057 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-dce93377-75a2-4a19-a821-053b7c547446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b0facee181c4109a33c701132ebe946'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:24:59.680981 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1123/2264] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:24:59 2022] DELETE /identity/v3/users/0b0facee181c4109a33c701132ebe946 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:25:00.014256 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e253a-1e46-48b7-aef1-30aca67331bb None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:00.014537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e253a-1e46-48b7-aef1-30aca67331bb None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:00.014827 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e253a-1e46-48b7-aef1-30aca67331bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:00.048446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-f24e253a-1e46-48b7-aef1-30aca67331bb None None] MFA Rules not processed for user `2ab18214df00412bb7440535ac725056`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:00.050615 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-f24e253a-1e46-48b7-aef1-30aca67331bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:00.123382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:00.144706 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce93377-75a2-4a19-a821-053b7c547446 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:00.146197 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-dce93377-75a2-4a19-a821-053b7c547446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:00.204662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1142/2265] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:25:00.253796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e253a-1e46-48b7-aef1-30aca67331bb None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:00.255422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-f24e253a-1e46-48b7-aef1-30aca67331bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:00.322251 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce93377-75a2-4a19-a821-053b7c547446 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:00.329649 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c357a34-1eae-41af-af53-9b1c1521c083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:00.330928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c357a34-1eae-41af-af53-9b1c1521c083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:00.331369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c357a34-1eae-41af-af53-9b1c1521c083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:00.331954 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c357a34-1eae-41af-af53-9b1c1521c083 admin admin] PATH_INFO: `/v3/projects/cae607c773ff4e2aaadf9fcc5160810f` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:00.334044 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c357a34-1eae-41af-af53-9b1c1521c083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cae607c773ff4e2aaadf9fcc5160810f)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:00.336153 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c357a34-1eae-41af-af53-9b1c1521c083 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:00.353517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e253a-1e46-48b7-aef1-30aca67331bb None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:00.359705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00aa543-ca66-4f4c-b109-5d0f7dca4b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:00.360663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00aa543-ca66-4f4c-b109-5d0f7dca4b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:00.362287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00aa543-ca66-4f4c-b109-5d0f7dca4b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:00.362738 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00aa543-ca66-4f4c-b109-5d0f7dca4b25 admin admin] PATH_INFO: `/v3/projects/a726cf9fcef74ff48ca9569b2e8d3023` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:00.364113 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00aa543-ca66-4f4c-b109-5d0f7dca4b25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a726cf9fcef74ff48ca9569b2e8d3023)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:00.367072 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00aa543-ca66-4f4c-b109-5d0f7dca4b25 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:00.405511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1c357a34-1eae-41af-af53-9b1c1521c083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cae607c773ff4e2aaadf9fcc5160810f', 'request_id': 'req-1c357a34-1eae-41af-af53-9b1c1521c083'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:00.421567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1c357a34-1eae-41af-af53-9b1c1521c083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cae607c773ff4e2aaadf9fcc5160810f', 'request_id': 'req-1c357a34-1eae-41af-af53-9b1c1521c083'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:00.436833 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1c357a34-1eae-41af-af53-9b1c1521c083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cae607c773ff4e2aaadf9fcc5160810f', 'request_id': 'req-1c357a34-1eae-41af-af53-9b1c1521c083'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:00.443181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d00aa543-ca66-4f4c-b109-5d0f7dca4b25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a726cf9fcef74ff48ca9569b2e8d3023', 'request_id': 'req-d00aa543-ca66-4f4c-b109-5d0f7dca4b25'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:00.455794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1c357a34-1eae-41af-af53-9b1c1521c083 admin admin] The token cache is being invalidate because project cae607c773ff4e2aaadf9fcc5160810f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:25:00.456182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1c357a34-1eae-41af-af53-9b1c1521c083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:00.457543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1c357a34-1eae-41af-af53-9b1c1521c083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:00.464879 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1124/2266] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:25:00 2022] DELETE /identity/v3/projects/cae607c773ff4e2aaadf9fcc5160810f => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:00.465675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d00aa543-ca66-4f4c-b109-5d0f7dca4b25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a726cf9fcef74ff48ca9569b2e8d3023', 'request_id': 'req-d00aa543-ca66-4f4c-b109-5d0f7dca4b25'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:00.466498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02f88c-f0a6-4915-8b8c-83d3d22f6670 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:00.466792 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02f88c-f0a6-4915-8b8c-83d3d22f6670 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:00.467068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02f88c-f0a6-4915-8b8c-83d3d22f6670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:00.479631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d00aa543-ca66-4f4c-b109-5d0f7dca4b25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a726cf9fcef74ff48ca9569b2e8d3023', 'request_id': 'req-d00aa543-ca66-4f4c-b109-5d0f7dca4b25'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:00.495617 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d00aa543-ca66-4f4c-b109-5d0f7dca4b25 admin admin] The token cache is being invalidate because project a726cf9fcef74ff48ca9569b2e8d3023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:25:00.496152 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d00aa543-ca66-4f4c-b109-5d0f7dca4b25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:00.497028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d00aa543-ca66-4f4c-b109-5d0f7dca4b25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:00.499691 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1143/2267] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:25:00 2022] DELETE /identity/v3/projects/a726cf9fcef74ff48ca9569b2e8d3023 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:25:00.500815 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-8a02f88c-f0a6-4915-8b8c-83d3d22f6670 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:00.502948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8a02f88c-f0a6-4915-8b8c-83d3d22f6670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:00.506563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28fa7b-43bd-4ec2-afee-8d2d56279d94 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:00.507143 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28fa7b-43bd-4ec2-afee-8d2d56279d94 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:00.507716 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28fa7b-43bd-4ec2-afee-8d2d56279d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:00.536108 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-aa28fa7b-43bd-4ec2-afee-8d2d56279d94 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:00.538105 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-aa28fa7b-43bd-4ec2-afee-8d2d56279d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:00.591831 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:00.622301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1125/2268] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:00.632761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6792c-16b2-4644-b863-f138b7976132 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:00.633194 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6792c-16b2-4644-b863-f138b7976132 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:00.634470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6792c-16b2-4644-b863-f138b7976132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:00.665145 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-f0b6792c-16b2-4644-b863-f138b7976132 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:00.667158 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f0b6792c-16b2-4644-b863-f138b7976132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:00.682051 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1126/2269] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:00.683026 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6792c-16b2-4644-b863-f138b7976132 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:00.688377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1144/2270] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:25:00.694637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6792c-16b2-4644-b863-f138b7976132 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:00.698189 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e32ef-6c12-43be-bca2-6735642e0364 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:00.698750 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e32ef-6c12-43be-bca2-6735642e0364 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:00.699304 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e32ef-6c12-43be-bca2-6735642e0364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:00.700384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a99ecb-e3b6-4eb0-acb1-ac1a9ee97136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:00.701050 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a99ecb-e3b6-4eb0-acb1-ac1a9ee97136 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:00.701372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a99ecb-e3b6-4eb0-acb1-ac1a9ee97136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:00.701641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a99ecb-e3b6-4eb0-acb1-ac1a9ee97136 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:00.702415 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a99ecb-e3b6-4eb0-acb1-ac1a9ee97136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:25:00.702720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a99ecb-e3b6-4eb0-acb1-ac1a9ee97136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:00.704719 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a99ecb-e3b6-4eb0-acb1-ac1a9ee97136 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:00.718408 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1127/2271] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:25:00.725548 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a99ecb-e3b6-4eb0-acb1-ac1a9ee97136 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:00.744993 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-0e0e32ef-6c12-43be-bca2-6735642e0364 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:00.747033 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-0e0e32ef-6c12-43be-bca2-6735642e0364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:00.748286 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a99ecb-e3b6-4eb0-acb1-ac1a9ee97136 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:00.754616 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0594cb3-5c1d-42f8-940a-73d725aa6c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:00.755525 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0594cb3-5c1d-42f8-940a-73d725aa6c27 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:00.755805 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0594cb3-5c1d-42f8-940a-73d725aa6c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:00.756081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0594cb3-5c1d-42f8-940a-73d725aa6c27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:00.757096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0594cb3-5c1d-42f8-940a-73d725aa6c27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:00.759424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0594cb3-5c1d-42f8-940a-73d725aa6c27 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:00.769941 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1145/2272] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:25:00.772624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e32ef-6c12-43be-bca2-6735642e0364 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:00.785111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e32ef-6c12-43be-bca2-6735642e0364 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:00.797247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daea322-1c49-409c-a929-99c3b0813343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:00.798578 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daea322-1c49-409c-a929-99c3b0813343 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:00.799900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daea322-1c49-409c-a929-99c3b0813343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:00.800290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daea322-1c49-409c-a929-99c3b0813343 admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:00.801069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daea322-1c49-409c-a929-99c3b0813343 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:25:00.802974 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1128/2273] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:25:00 2022] POST /identity/v3/projects => generated 377 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:00.806290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daea322-1c49-409c-a929-99c3b0813343 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:00.807156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0594cb3-5c1d-42f8-940a-73d725aa6c27 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:00.810088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daea322-1c49-409c-a929-99c3b0813343 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:00.828181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:00.859578 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1146/2274] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:25:00.860871 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daea322-1c49-409c-a929-99c3b0813343 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:00.883445 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daea322-1c49-409c-a929-99c3b0813343 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:00.910752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0594cb3-5c1d-42f8-940a-73d725aa6c27 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:00.916813 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba325d3-b45b-42b8-b774-42e666402269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:00.917642 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba325d3-b45b-42b8-b774-42e666402269 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:00.918020 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba325d3-b45b-42b8-b774-42e666402269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:00.919103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba325d3-b45b-42b8-b774-42e666402269 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:00.919900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba325d3-b45b-42b8-b774-42e666402269 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:00.922493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba325d3-b45b-42b8-b774-42e666402269 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:00.955724 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1129/2275] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:25:00 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:00.960329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba325d3-b45b-42b8-b774-42e666402269 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:00.978125 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba325d3-b45b-42b8-b774-42e666402269 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:00.984449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c555445-b54d-4a9a-9ced-5ff034b480f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:00.985511 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d6cb1-fddc-4868-adb4-f386e8ddf050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:00.985682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c555445-b54d-4a9a-9ced-5ff034b480f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:00.985871 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c555445-b54d-4a9a-9ced-5ff034b480f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:00.986453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d6cb1-fddc-4868-adb4-f386e8ddf050 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:00.986759 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c555445-b54d-4a9a-9ced-5ff034b480f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:00.986884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d6cb1-fddc-4868-adb4-f386e8ddf050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:00.987241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d6cb1-fddc-4868-adb4-f386e8ddf050 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:00.987912 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c555445-b54d-4a9a-9ced-5ff034b480f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:00.988040 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d6cb1-fddc-4868-adb4-f386e8ddf050 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:00.990206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c555445-b54d-4a9a-9ced-5ff034b480f6 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:00.991807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d6cb1-fddc-4868-adb4-f386e8ddf050 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:01.006521 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1130/2276] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:01.010504 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c555445-b54d-4a9a-9ced-5ff034b480f6 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:01.027795 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c555445-b54d-4a9a-9ced-5ff034b480f6 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:01.044752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1147/2277] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:25:00 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:01.046984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d6cb1-fddc-4868-adb4-f386e8ddf050 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:01.147379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6427e-07d6-41c2-b5b5-9b0905d6b6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:01.148156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6427e-07d6-41c2-b5b5-9b0905d6b6cb admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:01.148446 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6427e-07d6-41c2-b5b5-9b0905d6b6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:01.148723 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6427e-07d6-41c2-b5b5-9b0905d6b6cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:01.149455 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e6427e-07d6-41c2-b5b5-9b0905d6b6cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:01.151073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e6427e-07d6-41c2-b5b5-9b0905d6b6cb admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:01.163272 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1131/2278] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:01.169008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6427e-07d6-41c2-b5b5-9b0905d6b6cb admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:01.182534 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d6cb1-fddc-4868-adb4-f386e8ddf050 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:01.186493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a2fcf-d4d3-4f62-9db7-0cc34b1567f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:01.187635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a2fcf-d4d3-4f62-9db7-0cc34b1567f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:01.187992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a2fcf-d4d3-4f62-9db7-0cc34b1567f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:01.188697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a2fcf-d4d3-4f62-9db7-0cc34b1567f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:01.189104 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a2fcf-d4d3-4f62-9db7-0cc34b1567f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:01.191824 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a2fcf-d4d3-4f62-9db7-0cc34b1567f4 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:01.194963 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6427e-07d6-41c2-b5b5-9b0905d6b6cb admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:01.207982 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00746f-83de-4f00-bebd-a5ab1a873708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:01.208994 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00746f-83de-4f00-bebd-a5ab1a873708 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:01.209591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00746f-83de-4f00-bebd-a5ab1a873708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:01.209815 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00746f-83de-4f00-bebd-a5ab1a873708 admin admin] PATH_INFO: `/v3/projects/4b2020bddfbe4145936e64db6c1c8a06/users/17d7cab966f64b479831b4c57897f20a/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:01.228120 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1148/2279] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:25:01 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:01.234621 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a2fcf-d4d3-4f62-9db7-0cc34b1567f4 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:01.250954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a2fcf-d4d3-4f62-9db7-0cc34b1567f4 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:01.253968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00746f-83de-4f00-bebd-a5ab1a873708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b2020bddfbe4145936e64db6c1c8a06, user_id=17d7cab966f64b479831b4c57897f20a, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:01.256805 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00746f-83de-4f00-bebd-a5ab1a873708 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:01.266064 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b518d-7057-462a-a4a4-24dc7c6ac3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:01.267773 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b518d-7057-462a-a4a4-24dc7c6ac3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:01.268720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b518d-7057-462a-a4a4-24dc7c6ac3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:01.269391 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b518d-7057-462a-a4a4-24dc7c6ac3f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:01.270626 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b518d-7057-462a-a4a4-24dc7c6ac3f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:01.273839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b518d-7057-462a-a4a4-24dc7c6ac3f1 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:01.285929 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1132/2280] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:25:01 2022] PUT /identity/v3/projects/4b2020bddfbe4145936e64db6c1c8a06/users/17d7cab966f64b479831b4c57897f20a/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:01.287723 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4296d-9e30-4f45-acc8-e74450637f07 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:01.289506 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4296d-9e30-4f45-acc8-e74450637f07 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:01.290369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4296d-9e30-4f45-acc8-e74450637f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:01.304284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:01.315053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1149/2281] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:01.317440 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b518d-7057-462a-a4a4-24dc7c6ac3f1 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:01.332507 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b518d-7057-462a-a4a4-24dc7c6ac3f1 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:01.374345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-75b4296d-9e30-4f45-acc8-e74450637f07 None None] MFA Rules not processed for user `17d7cab966f64b479831b4c57897f20a`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:01.376266 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-75b4296d-9e30-4f45-acc8-e74450637f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:01.410446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362c9b6-3676-497e-b82c-a210bb685cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:01.411096 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362c9b6-3676-497e-b82c-a210bb685cee admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:01.411382 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362c9b6-3676-497e-b82c-a210bb685cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:01.411661 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362c9b6-3676-497e-b82c-a210bb685cee admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:01.412322 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362c9b6-3676-497e-b82c-a210bb685cee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:01.414353 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362c9b6-3676-497e-b82c-a210bb685cee admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:01.421779 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:01.435165 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1150/2282] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:01.436605 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362c9b6-3676-497e-b82c-a210bb685cee admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:01.453214 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362c9b6-3676-497e-b82c-a210bb685cee admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:01.473215 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2f54b-b222-46da-8976-522a6ca2d8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:01.473833 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2f54b-b222-46da-8976-522a6ca2d8ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:01.474145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2f54b-b222-46da-8976-522a6ca2d8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:01.474429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2f54b-b222-46da-8976-522a6ca2d8ce admin admin] PATH_INFO: `/v3/projects/f0ebb47b1a234f8d89649cc5db9a5750/users/908d1bcf05d7421eb91c65e826181ebd/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:01.494923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2f54b-b222-46da-8976-522a6ca2d8ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ebb47b1a234f8d89649cc5db9a5750, user_id=908d1bcf05d7421eb91c65e826181ebd, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:01.499735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2f54b-b222-46da-8976-522a6ca2d8ce admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:01.526630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1133/2283] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:01.529907 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1151/2284] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:25:01 2022] PUT /identity/v3/projects/f0ebb47b1a234f8d89649cc5db9a5750/users/908d1bcf05d7421eb91c65e826181ebd/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:01.540569 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4462e4-6513-439a-ae01-554ed750c63d None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:01.540901 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4462e4-6513-439a-ae01-554ed750c63d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:01.541239 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4462e4-6513-439a-ae01-554ed750c63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:01.546838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2f54b-b222-46da-8976-522a6ca2d8ce admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:01.552413 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5da2f54b-b222-46da-8976-522a6ca2d8ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:01.578896 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-cd4462e4-6513-439a-ae01-554ed750c63d None None] MFA Rules not processed for user `908d1bcf05d7421eb91c65e826181ebd`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:01.581909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-cd4462e4-6513-439a-ae01-554ed750c63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:01.687383 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2f54b-b222-46da-8976-522a6ca2d8ce admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:01.692566 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7ef2f-c48b-4798-aa52-f1181e5c5503 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e08e00bcb5f47ffab58974ba376938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:01.693735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7ef2f-c48b-4798-aa52-f1181e5c5503 service glance] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:01.694029 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7ef2f-c48b-4798-aa52-f1181e5c5503 service glance] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:01.694307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7ef2f-c48b-4798-aa52-f1181e5c5503 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:01.704219 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1134/2285] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:01.709916 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4462e4-6513-439a-ae01-554ed750c63d None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:01.824177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7ef2f-c48b-4798-aa52-f1181e5c5503 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:01.826252 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7ef2f-c48b-4798-aa52-f1181e5c5503 service glance] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:01.862288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1152/2286] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:25:01.878070 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4462e4-6513-439a-ae01-554ed750c63d None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:01.904369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140f500-37ef-4f81-88bd-68347144e923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:01.905270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140f500-37ef-4f81-88bd-68347144e923 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:01.905378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140f500-37ef-4f81-88bd-68347144e923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:01.905777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140f500-37ef-4f81-88bd-68347144e923 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:01.906396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140f500-37ef-4f81-88bd-68347144e923 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:01.909359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140f500-37ef-4f81-88bd-68347144e923 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:01.985462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1135/2287] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:25:01 2022] POST /identity/v3/projects => generated 387 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:01.990638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7ef2f-c48b-4798-aa52-f1181e5c5503 service glance] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:02.004114 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7ef2f-c48b-4798-aa52-f1181e5c5503 service glance] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:02.147968 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5e2b9-4b6c-47e8-9645-53aa1cb5d718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:02.148702 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5e2b9-4b6c-47e8-9645-53aa1cb5d718 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:02.149326 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5e2b9-4b6c-47e8-9645-53aa1cb5d718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:02.149623 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5e2b9-4b6c-47e8-9645-53aa1cb5d718 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:02.151389 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5e2b9-4b6c-47e8-9645-53aa1cb5d718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:02.153932 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5e2b9-4b6c-47e8-9645-53aa1cb5d718 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:02.187761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1153/2288] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:25:01 2022] POST /identity/v3/users => generated 300 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:02.192287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140f500-37ef-4f81-88bd-68347144e923 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:02.206272 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140f500-37ef-4f81-88bd-68347144e923 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:02.219093 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63022f63-8805-46ad-81ed-63ddc88e5213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:02.219731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63022f63-8805-46ad-81ed-63ddc88e5213 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:02.220013 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63022f63-8805-46ad-81ed-63ddc88e5213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:02.220291 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63022f63-8805-46ad-81ed-63ddc88e5213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:02.221008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63022f63-8805-46ad-81ed-63ddc88e5213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:02.223408 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63022f63-8805-46ad-81ed-63ddc88e5213 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:02.230918 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:02.272470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1136/2289] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:02.276520 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5e2b9-4b6c-47e8-9645-53aa1cb5d718 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:02.292215 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5e2b9-4b6c-47e8-9645-53aa1cb5d718 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:02.305450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5fa0d-3690-4d15-8d05-6281805561f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:02.306227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5fa0d-3690-4d15-8d05-6281805561f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:02.306577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5fa0d-3690-4d15-8d05-6281805561f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:02.306925 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5fa0d-3690-4d15-8d05-6281805561f2 admin admin] PATH_INFO: `/v3/projects/f59e818449be4da39738e8d4185101da/users/ae54b55fa64448f0984c082ab9557a6c/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:02.346365 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5fa0d-3690-4d15-8d05-6281805561f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f59e818449be4da39738e8d4185101da, user_id=ae54b55fa64448f0984c082ab9557a6c, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:02.354571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5fa0d-3690-4d15-8d05-6281805561f2 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:02.358487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:02.403037 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1154/2290] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:25:02 2022] PUT /identity/v3/projects/f59e818449be4da39738e8d4185101da/users/ae54b55fa64448f0984c082ab9557a6c/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:25:02.407204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63022f63-8805-46ad-81ed-63ddc88e5213 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:02.425730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63022f63-8805-46ad-81ed-63ddc88e5213 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:02.501071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3278d7-57d1-4f6c-b989-e00582351206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:02.501914 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3278d7-57d1-4f6c-b989-e00582351206 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:02.502321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3278d7-57d1-4f6c-b989-e00582351206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:02.502889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3278d7-57d1-4f6c-b989-e00582351206 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:02.503634 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3278d7-57d1-4f6c-b989-e00582351206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:02.505293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3278d7-57d1-4f6c-b989-e00582351206 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:02.521916 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1137/2291] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:02.522359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5fa0d-3690-4d15-8d05-6281805561f2 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:02.537564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5fa0d-3690-4d15-8d05-6281805561f2 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:02.549236 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed13f0-6a30-4d4e-af43-528c70e99e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:02.549995 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed13f0-6a30-4d4e-af43-528c70e99e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:02.550342 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed13f0-6a30-4d4e-af43-528c70e99e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:02.550683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed13f0-6a30-4d4e-af43-528c70e99e85 admin admin] PATH_INFO: `/v3/projects/f59e818449be4da39738e8d4185101da/users/ae54b55fa64448f0984c082ab9557a6c/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:02.552837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed13f0-6a30-4d4e-af43-528c70e99e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f59e818449be4da39738e8d4185101da, user_id=ae54b55fa64448f0984c082ab9557a6c, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:02.556795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed13f0-6a30-4d4e-af43-528c70e99e85 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:02.579112 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1155/2292] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:25:02 2022] PUT /identity/v3/projects/f59e818449be4da39738e8d4185101da/users/ae54b55fa64448f0984c082ab9557a6c/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:25:02.581360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccd557-3679-4bab-a7e6-21eface7b240 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:02.581644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccd557-3679-4bab-a7e6-21eface7b240 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:02.581926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccd557-3679-4bab-a7e6-21eface7b240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:02.619307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e2ccd557-3679-4bab-a7e6-21eface7b240 None None] MFA Rules not processed for user `ae54b55fa64448f0984c082ab9557a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:02.620516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e2ccd557-3679-4bab-a7e6-21eface7b240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:02.693851 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1138/2293] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:02.706265 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed13f0-6a30-4d4e-af43-528c70e99e85 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:02.708013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-02ed13f0-6a30-4d4e-af43-528c70e99e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:02.765703 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed13f0-6a30-4d4e-af43-528c70e99e85 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:02.769979 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2fdb7-bb27-453a-b025-a558eeead1e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:02.770603 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2fdb7-bb27-453a-b025-a558eeead1e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:02.770900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2fdb7-bb27-453a-b025-a558eeead1e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:02.771173 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2fdb7-bb27-453a-b025-a558eeead1e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:02.780615 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2fdb7-bb27-453a-b025-a558eeead1e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:02.782020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2fdb7-bb27-453a-b025-a558eeead1e6 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:02.793478 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1156/2294] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:25:02.824528 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccd557-3679-4bab-a7e6-21eface7b240 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:02.893915 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccd557-3679-4bab-a7e6-21eface7b240 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:02.898501 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414824ec-f3a6-4019-86ae-4d84e27a84bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:02.899131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414824ec-f3a6-4019-86ae-4d84e27a84bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:02.899416 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414824ec-f3a6-4019-86ae-4d84e27a84bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:02.899690 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414824ec-f3a6-4019-86ae-4d84e27a84bb admin admin] PATH_INFO: `/v3/users/908d1bcf05d7421eb91c65e826181ebd` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:02.900817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414824ec-f3a6-4019-86ae-4d84e27a84bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=908d1bcf05d7421eb91c65e826181ebd)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:02.902759 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414824ec-f3a6-4019-86ae-4d84e27a84bb admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:02.925794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:02.965945 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-414824ec-f3a6-4019-86ae-4d84e27a84bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '908d1bcf05d7421eb91c65e826181ebd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:02.966867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-414824ec-f3a6-4019-86ae-4d84e27a84bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '908d1bcf05d7421eb91c65e826181ebd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:02.981715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-414824ec-f3a6-4019-86ae-4d84e27a84bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '908d1bcf05d7421eb91c65e826181ebd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:02.991117 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-414824ec-f3a6-4019-86ae-4d84e27a84bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '908d1bcf05d7421eb91c65e826181ebd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:02.992061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-414824ec-f3a6-4019-86ae-4d84e27a84bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '908d1bcf05d7421eb91c65e826181ebd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:03.013984 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1139/2295] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:25:02 2022] DELETE /identity/v3/users/908d1bcf05d7421eb91c65e826181ebd => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:03.018875 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d02095-0158-4015-b4b0-68e656e32582 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:03.019154 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d02095-0158-4015-b4b0-68e656e32582 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:03.019367 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d02095-0158-4015-b4b0-68e656e32582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:03.041788 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-06d02095-0158-4015-b4b0-68e656e32582 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:03.043040 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-06d02095-0158-4015-b4b0-68e656e32582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:03.111089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1157/2296] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:25:03.121198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414824ec-f3a6-4019-86ae-4d84e27a84bb admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:03.122608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-414824ec-f3a6-4019-86ae-4d84e27a84bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:03.180223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414824ec-f3a6-4019-86ae-4d84e27a84bb admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:03.183900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fe4b3-fd9a-4715-8399-728fb3895dfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:03.184511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fe4b3-fd9a-4715-8399-728fb3895dfe service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:03.184794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fe4b3-fd9a-4715-8399-728fb3895dfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:03.185084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fe4b3-fd9a-4715-8399-728fb3895dfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:03.195284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662fe4b3-fd9a-4715-8399-728fb3895dfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:03.196713 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662fe4b3-fd9a-4715-8399-728fb3895dfe service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:03.206366 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1140/2297] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:25:03.481264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d02095-0158-4015-b4b0-68e656e32582 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:03.482735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-06d02095-0158-4015-b4b0-68e656e32582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:03.495811 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d02095-0158-4015-b4b0-68e656e32582 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:03.501585 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:03.503171 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:03.503442 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:03.503710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260 admin admin] PATH_INFO: `/v3/projects/f0ebb47b1a234f8d89649cc5db9a5750` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:03.504833 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0ebb47b1a234f8d89649cc5db9a5750)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:03.507563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:03.555709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0ebb47b1a234f8d89649cc5db9a5750', 'request_id': 'req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:03.563723 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0ebb47b1a234f8d89649cc5db9a5750', 'request_id': 'req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:03.572539 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0ebb47b1a234f8d89649cc5db9a5750', 'request_id': 'req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:03.584401 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260 admin admin] The token cache is being invalidate because project f0ebb47b1a234f8d89649cc5db9a5750 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:25:03.584719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:03.585393 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:03.586950 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1158/2298] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:25:03 2022] DELETE /identity/v3/projects/f0ebb47b1a234f8d89649cc5db9a5750 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:03.590933 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fe4b3-fd9a-4715-8399-728fb3895dfe service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:03.604193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662fe4b3-fd9a-4715-8399-728fb3895dfe service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:03.677749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978aa621-478a-4859-91d0-ddc5aa18c6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:03.678997 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978aa621-478a-4859-91d0-ddc5aa18c6c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:03.678997 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978aa621-478a-4859-91d0-ddc5aa18c6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:03.678997 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978aa621-478a-4859-91d0-ddc5aa18c6c9 admin admin] PATH_INFO: `/v3/users/ae54b55fa64448f0984c082ab9557a6c` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:03.680108 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978aa621-478a-4859-91d0-ddc5aa18c6c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae54b55fa64448f0984c082ab9557a6c)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:03.682119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978aa621-478a-4859-91d0-ddc5aa18c6c9 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:03.740670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:03.743986 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-978aa621-478a-4859-91d0-ddc5aa18c6c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae54b55fa64448f0984c082ab9557a6c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:03.745083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-978aa621-478a-4859-91d0-ddc5aa18c6c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae54b55fa64448f0984c082ab9557a6c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:03.760850 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-978aa621-478a-4859-91d0-ddc5aa18c6c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae54b55fa64448f0984c082ab9557a6c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:03.770737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-978aa621-478a-4859-91d0-ddc5aa18c6c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae54b55fa64448f0984c082ab9557a6c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:03.771363 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-978aa621-478a-4859-91d0-ddc5aa18c6c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae54b55fa64448f0984c082ab9557a6c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:03.785614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1141/2299] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:25:03 2022] DELETE /identity/v3/users/ae54b55fa64448f0984c082ab9557a6c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:04.061269 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:04.062726 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:04.084139 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc81bc-3c0e-4cb0-8b75-5954a3e85260 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:04.090685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:04.180419 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c171535-fe7e-48be-a8ff-e4cc586264b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:04.181163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c171535-fe7e-48be-a8ff-e4cc586264b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:04.181518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c171535-fe7e-48be-a8ff-e4cc586264b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:04.181857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c171535-fe7e-48be-a8ff-e4cc586264b6 admin admin] PATH_INFO: `/v3/projects/f59e818449be4da39738e8d4185101da` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:04.183098 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c171535-fe7e-48be-a8ff-e4cc586264b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f59e818449be4da39738e8d4185101da)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:04.189332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c171535-fe7e-48be-a8ff-e4cc586264b6 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:04.241854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8c171535-fe7e-48be-a8ff-e4cc586264b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f59e818449be4da39738e8d4185101da', 'request_id': 'req-8c171535-fe7e-48be-a8ff-e4cc586264b6'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:04.248596 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8c171535-fe7e-48be-a8ff-e4cc586264b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f59e818449be4da39738e8d4185101da', 'request_id': 'req-8c171535-fe7e-48be-a8ff-e4cc586264b6'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:04.260513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8c171535-fe7e-48be-a8ff-e4cc586264b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f59e818449be4da39738e8d4185101da', 'request_id': 'req-8c171535-fe7e-48be-a8ff-e4cc586264b6'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:04.272274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8c171535-fe7e-48be-a8ff-e4cc586264b6 admin admin] The token cache is being invalidate because project f59e818449be4da39738e8d4185101da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:25:04.272597 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8c171535-fe7e-48be-a8ff-e4cc586264b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:04.273273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8c171535-fe7e-48be-a8ff-e4cc586264b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:04.274757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1159/2300] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:25:04 2022] DELETE /identity/v3/projects/f59e818449be4da39738e8d4185101da => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:25:04.279648 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4db253-670f-4d40-b176-ab6c3b5c33e7 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:04.280002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4db253-670f-4d40-b176-ab6c3b5c33e7 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:04.280372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4db253-670f-4d40-b176-ab6c3b5c33e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:04.301583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-0b4db253-670f-4d40-b176-ab6c3b5c33e7 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:04.302785 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0b4db253-670f-4d40-b176-ab6c3b5c33e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:04.372880 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1142/2301] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:25:04.381689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e955496-08b2-43bc-96af-d80a29825c82 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:04.381982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e955496-08b2-43bc-96af-d80a29825c82 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:04.382264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e955496-08b2-43bc-96af-d80a29825c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:04.403583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-1e955496-08b2-43bc-96af-d80a29825c82 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:04.404813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-1e955496-08b2-43bc-96af-d80a29825c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:04.414090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1160/2302] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:04.418592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4db253-670f-4d40-b176-ab6c3b5c33e7 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:04.433888 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4db253-670f-4d40-b176-ab6c3b5c33e7 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:04.438701 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9388e5a-a37d-4025-a05e-d8e252d0b77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:04.439477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9388e5a-a37d-4025-a05e-d8e252d0b77d admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:04.439889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9388e5a-a37d-4025-a05e-d8e252d0b77d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:04.440219 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9388e5a-a37d-4025-a05e-d8e252d0b77d admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:04.441067 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9388e5a-a37d-4025-a05e-d8e252d0b77d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:25:04.441407 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9388e5a-a37d-4025-a05e-d8e252d0b77d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:04.443021 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9388e5a-a37d-4025-a05e-d8e252d0b77d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:04.455492 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1143/2303] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:25:04.461851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e955496-08b2-43bc-96af-d80a29825c82 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:04.472894 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e955496-08b2-43bc-96af-d80a29825c82 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:04.479235 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ebde3-5787-464c-8144-8c713b7b7c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:04.479908 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ebde3-5787-464c-8144-8c713b7b7c3f admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:04.480256 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ebde3-5787-464c-8144-8c713b7b7c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:04.480594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ebde3-5787-464c-8144-8c713b7b7c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:04.481320 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ebde3-5787-464c-8144-8c713b7b7c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:04.483625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ebde3-5787-464c-8144-8c713b7b7c3f admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:04.518071 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1161/2304] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:25:04 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:04.522336 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9388e5a-a37d-4025-a05e-d8e252d0b77d admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:04.532391 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9388e5a-a37d-4025-a05e-d8e252d0b77d admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:04.594698 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7ac3f-3eba-48a6-8d17-fd255de559b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:04.595370 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7ac3f-3eba-48a6-8d17-fd255de559b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:04.596162 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7ac3f-3eba-48a6-8d17-fd255de559b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:04.596579 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a7ac3f-3eba-48a6-8d17-fd255de559b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:04.597737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7ac3f-3eba-48a6-8d17-fd255de559b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:04.599698 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a7ac3f-3eba-48a6-8d17-fd255de559b3 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:04.625509 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1144/2305] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:25:04 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:04.626946 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ebde3-5787-464c-8144-8c713b7b7c3f admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:04.637576 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ebde3-5787-464c-8144-8c713b7b7c3f admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:04.642881 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737eb64-86a2-42e1-baed-0b07ea6a1a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:04.643496 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737eb64-86a2-42e1-baed-0b07ea6a1a25 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:04.643772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737eb64-86a2-42e1-baed-0b07ea6a1a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:04.644042 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737eb64-86a2-42e1-baed-0b07ea6a1a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:04.644683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737eb64-86a2-42e1-baed-0b07ea6a1a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:04.646327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c737eb64-86a2-42e1-baed-0b07ea6a1a25 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:04.656638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1162/2306] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:04.661435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7ac3f-3eba-48a6-8d17-fd255de559b3 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:04.671619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a7ac3f-3eba-48a6-8d17-fd255de559b3 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:04.676286 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575cf3c-63ae-48ac-9aea-5dcb77640e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:04.676884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575cf3c-63ae-48ac-9aea-5dcb77640e58 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:04.677191 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575cf3c-63ae-48ac-9aea-5dcb77640e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:04.677466 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575cf3c-63ae-48ac-9aea-5dcb77640e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:04.678112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575cf3c-63ae-48ac-9aea-5dcb77640e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:04.679757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575cf3c-63ae-48ac-9aea-5dcb77640e58 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:04.690191 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1145/2307] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:04.695066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737eb64-86a2-42e1-baed-0b07ea6a1a25 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:04.706221 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737eb64-86a2-42e1-baed-0b07ea6a1a25 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:04.711620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1a22e-6abc-4696-b981-8a3ef5d2e33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:04.712325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1a22e-6abc-4696-b981-8a3ef5d2e33c admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:04.712733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1a22e-6abc-4696-b981-8a3ef5d2e33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:04.713025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1a22e-6abc-4696-b981-8a3ef5d2e33c admin admin] PATH_INFO: `/v3/projects/96021b3373b44b1fb8b02b852d04e9c6/users/a929e4ec9c96451198027aee014c6ca0/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:04.730952 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1a22e-6abc-4696-b981-8a3ef5d2e33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=96021b3373b44b1fb8b02b852d04e9c6, user_id=a929e4ec9c96451198027aee014c6ca0, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:04.734056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1a22e-6abc-4696-b981-8a3ef5d2e33c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:04.748386 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1163/2308] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:25:04 2022] PUT /identity/v3/projects/96021b3373b44b1fb8b02b852d04e9c6/users/a929e4ec9c96451198027aee014c6ca0/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:04.758739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f368023-6b79-4770-8302-338b8abbb85e None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:04.759135 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f368023-6b79-4770-8302-338b8abbb85e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:04.759413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f368023-6b79-4770-8302-338b8abbb85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:04.789157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-1f368023-6b79-4770-8302-338b8abbb85e None None] MFA Rules not processed for user `a929e4ec9c96451198027aee014c6ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:04.790385 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1f368023-6b79-4770-8302-338b8abbb85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:04.806910 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:04.865732 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1146/2309] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:04.870500 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1a22e-6abc-4696-b981-8a3ef5d2e33c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:04.914523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:04.939852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1a22e-6abc-4696-b981-8a3ef5d2e33c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:04.944514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65287344-f706-4a5f-98e2-a5acc4535040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:04.945120 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65287344-f706-4a5f-98e2-a5acc4535040 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:04.945421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65287344-f706-4a5f-98e2-a5acc4535040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:04.945677 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65287344-f706-4a5f-98e2-a5acc4535040 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:04.946328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65287344-f706-4a5f-98e2-a5acc4535040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:04.948395 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65287344-f706-4a5f-98e2-a5acc4535040 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:04.979432 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1164/2310] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:25:04 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:04.983421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f368023-6b79-4770-8302-338b8abbb85e None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:04.994193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f368023-6b79-4770-8302-338b8abbb85e None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:05.052283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f616c73-73f9-498b-9644-6a0d9902bf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:05.052928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f616c73-73f9-498b-9644-6a0d9902bf67 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:05.053240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f616c73-73f9-498b-9644-6a0d9902bf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:05.053517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f616c73-73f9-498b-9644-6a0d9902bf67 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:05.054160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f616c73-73f9-498b-9644-6a0d9902bf67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:05.056102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f616c73-73f9-498b-9644-6a0d9902bf67 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:05.078445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1147/2311] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:25:04 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:05.083176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65287344-f706-4a5f-98e2-a5acc4535040 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:05.093482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65287344-f706-4a5f-98e2-a5acc4535040 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:05.098678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5d23f-fa3c-4a28-a65c-3a15dee8f802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:05.099290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d23f-fa3c-4a28-a65c-3a15dee8f802 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:05.099571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d23f-fa3c-4a28-a65c-3a15dee8f802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:05.099845 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d23f-fa3c-4a28-a65c-3a15dee8f802 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:05.100497 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5d23f-fa3c-4a28-a65c-3a15dee8f802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:05.102117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5d23f-fa3c-4a28-a65c-3a15dee8f802 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:05.112999 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1165/2312] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:05.117108 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f616c73-73f9-498b-9644-6a0d9902bf67 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:05.128379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f616c73-73f9-498b-9644-6a0d9902bf67 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:05.133007 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769ad42-31dd-4bb8-ae6a-5af0b91eaa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:05.133711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769ad42-31dd-4bb8-ae6a-5af0b91eaa19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:05.133987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769ad42-31dd-4bb8-ae6a-5af0b91eaa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:05.134256 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769ad42-31dd-4bb8-ae6a-5af0b91eaa19 admin admin] PATH_INFO: `/v3/projects/66b5335c03014faeae5f66a0a11d3971/users/cbf9a2372c3c4f11845ed89d4604a017/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:05.150818 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769ad42-31dd-4bb8-ae6a-5af0b91eaa19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b5335c03014faeae5f66a0a11d3971, user_id=cbf9a2372c3c4f11845ed89d4604a017, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:05.153489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769ad42-31dd-4bb8-ae6a-5af0b91eaa19 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:05.167505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1148/2313] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:25:05 2022] PUT /identity/v3/projects/66b5335c03014faeae5f66a0a11d3971/users/cbf9a2372c3c4f11845ed89d4604a017/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:05.171533 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5d23f-fa3c-4a28-a65c-3a15dee8f802 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:05.185197 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5d23f-fa3c-4a28-a65c-3a15dee8f802 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:05.259151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b5830-593d-4f34-b413-b491cee73efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:05.259927 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b5830-593d-4f34-b413-b491cee73efa admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:05.260267 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b5830-593d-4f34-b413-b491cee73efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:05.260630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b5830-593d-4f34-b413-b491cee73efa admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:05.261375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b5830-593d-4f34-b413-b491cee73efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:05.263744 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b5830-593d-4f34-b413-b491cee73efa admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:05.275491 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1166/2314] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:25:05.279766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769ad42-31dd-4bb8-ae6a-5af0b91eaa19 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:05.292199 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769ad42-31dd-4bb8-ae6a-5af0b91eaa19 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:05.297594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac4ade-4da8-481b-8d6c-f6dedf9afd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:05.298942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac4ade-4da8-481b-8d6c-f6dedf9afd0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:05.299317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac4ade-4da8-481b-8d6c-f6dedf9afd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:05.299725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac4ade-4da8-481b-8d6c-f6dedf9afd0f admin admin] PATH_INFO: `/v3/projects/66b5335c03014faeae5f66a0a11d3971/users/cbf9a2372c3c4f11845ed89d4604a017/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:05.303811 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac4ade-4da8-481b-8d6c-f6dedf9afd0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b5335c03014faeae5f66a0a11d3971, user_id=cbf9a2372c3c4f11845ed89d4604a017, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:05.310308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac4ade-4da8-481b-8d6c-f6dedf9afd0f admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:05.324818 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1149/2315] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:25:05 2022] PUT /identity/v3/projects/66b5335c03014faeae5f66a0a11d3971/users/cbf9a2372c3c4f11845ed89d4604a017/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:05.339606 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975395fb-b713-41c3-95d0-933a812b043c None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:05.339934 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975395fb-b713-41c3-95d0-933a812b043c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:05.340215 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975395fb-b713-41c3-95d0-933a812b043c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:05.405044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-975395fb-b713-41c3-95d0-933a812b043c None None] MFA Rules not processed for user `cbf9a2372c3c4f11845ed89d4604a017`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:05.406626 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-975395fb-b713-41c3-95d0-933a812b043c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:05.507590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1167/2316] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:05.520065 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac4ade-4da8-481b-8d6c-f6dedf9afd0f admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:05.522008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-79ac4ade-4da8-481b-8d6c-f6dedf9afd0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:05.582742 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac4ade-4da8-481b-8d6c-f6dedf9afd0f admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:05.586598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dc592f-4236-4ba5-a0aa-ab9f196ce047 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:05.587210 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc592f-4236-4ba5-a0aa-ab9f196ce047 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:05.587492 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc592f-4236-4ba5-a0aa-ab9f196ce047 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:05.587765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dc592f-4236-4ba5-a0aa-ab9f196ce047 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:05.598702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc592f-4236-4ba5-a0aa-ab9f196ce047 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:05.600151 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dc592f-4236-4ba5-a0aa-ab9f196ce047 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:05.610286 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1150/2317] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:25:06.119691 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975395fb-b713-41c3-95d0-933a812b043c None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:06.154337 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:06.186614 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975395fb-b713-41c3-95d0-933a812b043c None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:06.191448 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c42d3-21d6-4e09-8479-421c00954f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:06.192083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c42d3-21d6-4e09-8479-421c00954f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:06.192365 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c42d3-21d6-4e09-8479-421c00954f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:06.192632 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c42d3-21d6-4e09-8479-421c00954f92 admin admin] PATH_INFO: `/v3/users/a929e4ec9c96451198027aee014c6ca0` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:06.193753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c42d3-21d6-4e09-8479-421c00954f92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a929e4ec9c96451198027aee014c6ca0)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:06.195638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c42d3-21d6-4e09-8479-421c00954f92 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:06.240635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-338c42d3-21d6-4e09-8479-421c00954f92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a929e4ec9c96451198027aee014c6ca0'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:06.241278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-338c42d3-21d6-4e09-8479-421c00954f92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a929e4ec9c96451198027aee014c6ca0'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:06.248416 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-338c42d3-21d6-4e09-8479-421c00954f92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a929e4ec9c96451198027aee014c6ca0'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:06.248971 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-338c42d3-21d6-4e09-8479-421c00954f92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a929e4ec9c96451198027aee014c6ca0'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:06.256370 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-338c42d3-21d6-4e09-8479-421c00954f92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a929e4ec9c96451198027aee014c6ca0'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:06.276545 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1168/2318] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:25:06 2022] DELETE /identity/v3/users/a929e4ec9c96451198027aee014c6ca0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:06.280956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffa513-a62a-4f4b-a689-a95fe445748e None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:06.281274 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffa513-a62a-4f4b-a689-a95fe445748e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:06.281556 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffa513-a62a-4f4b-a689-a95fe445748e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:06.302422 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-97ffa513-a62a-4f4b-a689-a95fe445748e None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:06.303636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-97ffa513-a62a-4f4b-a689-a95fe445748e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:06.308678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:06.366260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1151/2319] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:06.376376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c42d3-21d6-4e09-8479-421c00954f92 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:06.379022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-338c42d3-21d6-4e09-8479-421c00954f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:06.439632 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c42d3-21d6-4e09-8479-421c00954f92 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:06.444579 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47faba-54d1-42b8-862b-8ae4f57fa9d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:06.445237 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47faba-54d1-42b8-862b-8ae4f57fa9d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:06.445521 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47faba-54d1-42b8-862b-8ae4f57fa9d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:06.445798 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47faba-54d1-42b8-862b-8ae4f57fa9d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:06.455535 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47faba-54d1-42b8-862b-8ae4f57fa9d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:06.457255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47faba-54d1-42b8-862b-8ae4f57fa9d5 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:06.466952 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1169/2320] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:25:06.788145 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ffa513-a62a-4f4b-a689-a95fe445748e None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:06.789598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-97ffa513-a62a-4f4b-a689-a95fe445748e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:06.803378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ffa513-a62a-4f4b-a689-a95fe445748e None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:06.808870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41eb79-08f7-4939-acdd-f8391e78fe66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:06.809566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41eb79-08f7-4939-acdd-f8391e78fe66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:06.809851 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41eb79-08f7-4939-acdd-f8391e78fe66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:06.810128 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41eb79-08f7-4939-acdd-f8391e78fe66 admin admin] PATH_INFO: `/v3/projects/96021b3373b44b1fb8b02b852d04e9c6` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:06.811281 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41eb79-08f7-4939-acdd-f8391e78fe66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96021b3373b44b1fb8b02b852d04e9c6)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:06.813201 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41eb79-08f7-4939-acdd-f8391e78fe66 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:06.869360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8a41eb79-08f7-4939-acdd-f8391e78fe66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96021b3373b44b1fb8b02b852d04e9c6', 'request_id': 'req-8a41eb79-08f7-4939-acdd-f8391e78fe66'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:06.875644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8a41eb79-08f7-4939-acdd-f8391e78fe66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96021b3373b44b1fb8b02b852d04e9c6', 'request_id': 'req-8a41eb79-08f7-4939-acdd-f8391e78fe66'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:06.883711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8a41eb79-08f7-4939-acdd-f8391e78fe66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96021b3373b44b1fb8b02b852d04e9c6', 'request_id': 'req-8a41eb79-08f7-4939-acdd-f8391e78fe66'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:06.895621 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8a41eb79-08f7-4939-acdd-f8391e78fe66 admin admin] The token cache is being invalidate because project 96021b3373b44b1fb8b02b852d04e9c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:25:06.895911 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8a41eb79-08f7-4939-acdd-f8391e78fe66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:06.896554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8a41eb79-08f7-4939-acdd-f8391e78fe66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:06.898099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1152/2321] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:25:06 2022] DELETE /identity/v3/projects/96021b3373b44b1fb8b02b852d04e9c6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:06.902531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47faba-54d1-42b8-862b-8ae4f57fa9d5 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:06.912998 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47faba-54d1-42b8-862b-8ae4f57fa9d5 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:06.960331 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:06.970290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d169789-63ce-493c-811f-062803de54ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:06.970947 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d169789-63ce-493c-811f-062803de54ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:06.971254 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d169789-63ce-493c-811f-062803de54ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:06.971607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d169789-63ce-493c-811f-062803de54ff admin admin] PATH_INFO: `/v3/users/cbf9a2372c3c4f11845ed89d4604a017` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:06.973899 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d169789-63ce-493c-811f-062803de54ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbf9a2372c3c4f11845ed89d4604a017)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:06.976006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d169789-63ce-493c-811f-062803de54ff admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:07.024415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-4d169789-63ce-493c-811f-062803de54ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbf9a2372c3c4f11845ed89d4604a017'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:07.025120 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-4d169789-63ce-493c-811f-062803de54ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbf9a2372c3c4f11845ed89d4604a017'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:07.033219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-4d169789-63ce-493c-811f-062803de54ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbf9a2372c3c4f11845ed89d4604a017'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:07.033927 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-4d169789-63ce-493c-811f-062803de54ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbf9a2372c3c4f11845ed89d4604a017'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:07.042743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-4d169789-63ce-493c-811f-062803de54ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbf9a2372c3c4f11845ed89d4604a017'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:07.064079 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1170/2322] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:25:06 2022] DELETE /identity/v3/users/cbf9a2372c3c4f11845ed89d4604a017 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:25:07.407252 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41eb79-08f7-4939-acdd-f8391e78fe66 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:07.409367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8a41eb79-08f7-4939-acdd-f8391e78fe66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:07.430823 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41eb79-08f7-4939-acdd-f8391e78fe66 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:07.508438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358a6d6-0879-4ba9-9aba-e000e0162ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:07.510095 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358a6d6-0879-4ba9-9aba-e000e0162ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:07.510095 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358a6d6-0879-4ba9-9aba-e000e0162ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:07.510095 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358a6d6-0879-4ba9-9aba-e000e0162ccc admin admin] PATH_INFO: `/v3/projects/66b5335c03014faeae5f66a0a11d3971` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:07.511343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358a6d6-0879-4ba9-9aba-e000e0162ccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=66b5335c03014faeae5f66a0a11d3971)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:07.517166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358a6d6-0879-4ba9-9aba-e000e0162ccc admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:07.564019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:07.578143 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-9358a6d6-0879-4ba9-9aba-e000e0162ccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66b5335c03014faeae5f66a0a11d3971', 'request_id': 'req-9358a6d6-0879-4ba9-9aba-e000e0162ccc'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:07.589720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-9358a6d6-0879-4ba9-9aba-e000e0162ccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66b5335c03014faeae5f66a0a11d3971', 'request_id': 'req-9358a6d6-0879-4ba9-9aba-e000e0162ccc'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:07.599563 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-9358a6d6-0879-4ba9-9aba-e000e0162ccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66b5335c03014faeae5f66a0a11d3971', 'request_id': 'req-9358a6d6-0879-4ba9-9aba-e000e0162ccc'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:07.610515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-9358a6d6-0879-4ba9-9aba-e000e0162ccc admin admin] The token cache is being invalidate because project 66b5335c03014faeae5f66a0a11d3971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:25:07.610804 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-9358a6d6-0879-4ba9-9aba-e000e0162ccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:07.611430 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-9358a6d6-0879-4ba9-9aba-e000e0162ccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:07.617069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1153/2323] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:25:07 2022] DELETE /identity/v3/projects/66b5335c03014faeae5f66a0a11d3971 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:07.619056 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552d108-37fe-458d-9d61-bc7989465b99 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:07.619548 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552d108-37fe-458d-9d61-bc7989465b99 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:07.619866 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552d108-37fe-458d-9d61-bc7989465b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:07.640068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7552d108-37fe-458d-9d61-bc7989465b99 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:07.641299 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7552d108-37fe-458d-9d61-bc7989465b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:07.702448 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1171/2324] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:07.712384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a780b-e20e-43a3-b27e-a9984ac8fdad None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:07.712678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a780b-e20e-43a3-b27e-a9984ac8fdad None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:07.712956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a780b-e20e-43a3-b27e-a9984ac8fdad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:07.733302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-235a780b-e20e-43a3-b27e-a9984ac8fdad None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:07.734555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-235a780b-e20e-43a3-b27e-a9984ac8fdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:07.743587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1154/2325] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:07.747825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552d108-37fe-458d-9d61-bc7989465b99 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:07.757884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552d108-37fe-458d-9d61-bc7989465b99 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:07.763748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ea834-34c2-466d-92d0-c58c27f61f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:07.763748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ea834-34c2-466d-92d0-c58c27f61f1a admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:07.763748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ea834-34c2-466d-92d0-c58c27f61f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:07.763748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ea834-34c2-466d-92d0-c58c27f61f1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:07.764313 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ea834-34c2-466d-92d0-c58c27f61f1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:25:07.764618 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ea834-34c2-466d-92d0-c58c27f61f1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:07.766267 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ea834-34c2-466d-92d0-c58c27f61f1a admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:07.782519 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1172/2326] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:25:07.788340 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a780b-e20e-43a3-b27e-a9984ac8fdad None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:07.797903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a780b-e20e-43a3-b27e-a9984ac8fdad None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:07.802957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fd1cc-f748-43ca-93b5-7f308031f55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:07.803502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fd1cc-f748-43ca-93b5-7f308031f55e admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:07.803780 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fd1cc-f748-43ca-93b5-7f308031f55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:07.804052 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16fd1cc-f748-43ca-93b5-7f308031f55e admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:07.804686 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16fd1cc-f748-43ca-93b5-7f308031f55e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:07.806822 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16fd1cc-f748-43ca-93b5-7f308031f55e admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:07.838409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1155/2327] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:25:07 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:07.839723 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ea834-34c2-466d-92d0-c58c27f61f1a admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:07.850759 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ea834-34c2-466d-92d0-c58c27f61f1a admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:07.929348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded73e7f-9911-42ae-a477-a2ef68c9b5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:07.929348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded73e7f-9911-42ae-a477-a2ef68c9b5f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:07.929348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded73e7f-9911-42ae-a477-a2ef68c9b5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:07.929348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded73e7f-9911-42ae-a477-a2ef68c9b5f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:07.929807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded73e7f-9911-42ae-a477-a2ef68c9b5f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:07.929807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded73e7f-9911-42ae-a477-a2ef68c9b5f9 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:07.934506 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:07.949930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1173/2328] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:25:07 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:07.954502 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fd1cc-f748-43ca-93b5-7f308031f55e admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:07.965530 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16fd1cc-f748-43ca-93b5-7f308031f55e admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:07.970132 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40580264-4101-47f3-b94c-764d40af716f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:07.970726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580264-4101-47f3-b94c-764d40af716f admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:07.971006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580264-4101-47f3-b94c-764d40af716f admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:07.971280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40580264-4101-47f3-b94c-764d40af716f admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:07.972593 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40580264-4101-47f3-b94c-764d40af716f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:07.974223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40580264-4101-47f3-b94c-764d40af716f admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:07.983969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1156/2329] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:07.988105 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded73e7f-9911-42ae-a477-a2ef68c9b5f9 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:07.998122 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded73e7f-9911-42ae-a477-a2ef68c9b5f9 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:08.002777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa20be-792e-4d30-b403-10f360b5f6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:08.004220 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa20be-792e-4d30-b403-10f360b5f6be admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:08.004680 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa20be-792e-4d30-b403-10f360b5f6be admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:08.004961 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa20be-792e-4d30-b403-10f360b5f6be admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:08.005653 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa20be-792e-4d30-b403-10f360b5f6be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:08.007238 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa20be-792e-4d30-b403-10f360b5f6be admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:08.017655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1174/2330] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:08.022392 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40580264-4101-47f3-b94c-764d40af716f admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:08.032217 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40580264-4101-47f3-b94c-764d40af716f admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:08.036780 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08028c77-0d55-4de7-b20e-0b40c9fb4637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:08.037453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08028c77-0d55-4de7-b20e-0b40c9fb4637 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:08.037738 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08028c77-0d55-4de7-b20e-0b40c9fb4637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:08.038010 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08028c77-0d55-4de7-b20e-0b40c9fb4637 admin admin] PATH_INFO: `/v3/projects/4ab2fa46e2234901b2eecb6884dff703/users/a17f4c5306e749fea1d634be27287e25/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:08.053360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08028c77-0d55-4de7-b20e-0b40c9fb4637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ab2fa46e2234901b2eecb6884dff703, user_id=a17f4c5306e749fea1d634be27287e25, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:08.055957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08028c77-0d55-4de7-b20e-0b40c9fb4637 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:08.069106 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1157/2331] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:25:08 2022] PUT /identity/v3/projects/4ab2fa46e2234901b2eecb6884dff703/users/a17f4c5306e749fea1d634be27287e25/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:08.079152 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3260dad-9392-4485-81c9-8f0818529b7a None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:08.079439 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3260dad-9392-4485-81c9-8f0818529b7a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:08.079745 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3260dad-9392-4485-81c9-8f0818529b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:08.107379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-a3260dad-9392-4485-81c9-8f0818529b7a None None] MFA Rules not processed for user `a17f4c5306e749fea1d634be27287e25`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:08.108600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a3260dad-9392-4485-81c9-8f0818529b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:08.171484 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1175/2332] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:08.175940 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08028c77-0d55-4de7-b20e-0b40c9fb4637 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:08.237839 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08028c77-0d55-4de7-b20e-0b40c9fb4637 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:08.242615 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45fba8-1b45-437f-ba2f-13c81c45d58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:08.243219 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45fba8-1b45-437f-ba2f-13c81c45d58e admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:08.243513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45fba8-1b45-437f-ba2f-13c81c45d58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:08.243787 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff45fba8-1b45-437f-ba2f-13c81c45d58e admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:08.244450 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45fba8-1b45-437f-ba2f-13c81c45d58e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:08.246428 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff45fba8-1b45-437f-ba2f-13c81c45d58e admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:08.274257 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1158/2333] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:25:08 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:08.278381 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3260dad-9392-4485-81c9-8f0818529b7a None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:08.289384 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3260dad-9392-4485-81c9-8f0818529b7a None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:08.353159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c74e2-ebdc-47b9-b67a-d85baacc821b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:08.353801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c74e2-ebdc-47b9-b67a-d85baacc821b admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:08.354084 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c74e2-ebdc-47b9-b67a-d85baacc821b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:08.354468 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c74e2-ebdc-47b9-b67a-d85baacc821b admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:08.354998 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c74e2-ebdc-47b9-b67a-d85baacc821b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:08.356899 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c74e2-ebdc-47b9-b67a-d85baacc821b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:08.380057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1176/2334] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:25:08 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:08.384268 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45fba8-1b45-437f-ba2f-13c81c45d58e admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:08.395061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff45fba8-1b45-437f-ba2f-13c81c45d58e admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:08.399676 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a369c69-c665-4127-b0cc-7a059475c6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:08.400301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a369c69-c665-4127-b0cc-7a059475c6a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:08.400545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a369c69-c665-4127-b0cc-7a059475c6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:08.400818 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a369c69-c665-4127-b0cc-7a059475c6a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:08.401505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a369c69-c665-4127-b0cc-7a059475c6a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:08.403088 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a369c69-c665-4127-b0cc-7a059475c6a5 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:08.414631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1159/2335] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:08.418938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c74e2-ebdc-47b9-b67a-d85baacc821b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:08.429871 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c74e2-ebdc-47b9-b67a-d85baacc821b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:08.434408 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d897a5f-7b26-4ebc-a74d-7b2d9ecc508b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:08.435055 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d897a5f-7b26-4ebc-a74d-7b2d9ecc508b admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:08.435331 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d897a5f-7b26-4ebc-a74d-7b2d9ecc508b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:08.435604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d897a5f-7b26-4ebc-a74d-7b2d9ecc508b admin admin] PATH_INFO: `/v3/projects/b11c48638c0245779873bdadaec0f741/users/c1f8d61ec3e1426792000eed76c50a52/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:08.452299 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d897a5f-7b26-4ebc-a74d-7b2d9ecc508b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b11c48638c0245779873bdadaec0f741, user_id=c1f8d61ec3e1426792000eed76c50a52, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:08.454953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d897a5f-7b26-4ebc-a74d-7b2d9ecc508b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:08.468838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1177/2336] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:25:08 2022] PUT /identity/v3/projects/b11c48638c0245779873bdadaec0f741/users/c1f8d61ec3e1426792000eed76c50a52/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:25:08.473463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a369c69-c665-4127-b0cc-7a059475c6a5 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:08.485663 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a369c69-c665-4127-b0cc-7a059475c6a5 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:08.521114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:08.550903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86805c-e841-4ded-9f93-3e0a58c96263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:08.551508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86805c-e841-4ded-9f93-3e0a58c96263 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:08.551787 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86805c-e841-4ded-9f93-3e0a58c96263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:08.552055 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86805c-e841-4ded-9f93-3e0a58c96263 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:08.552692 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a86805c-e841-4ded-9f93-3e0a58c96263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:08.554337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a86805c-e841-4ded-9f93-3e0a58c96263 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:08.568872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1160/2337] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:08.572922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d897a5f-7b26-4ebc-a74d-7b2d9ecc508b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:08.584174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d897a5f-7b26-4ebc-a74d-7b2d9ecc508b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:08.588932 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d0da1-adc9-49e6-ad96-fab5b6f27cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:08.589700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d0da1-adc9-49e6-ad96-fab5b6f27cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:08.589978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d0da1-adc9-49e6-ad96-fab5b6f27cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:08.590251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d0da1-adc9-49e6-ad96-fab5b6f27cbc admin admin] PATH_INFO: `/v3/projects/b11c48638c0245779873bdadaec0f741/users/c1f8d61ec3e1426792000eed76c50a52/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:08.592167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d0da1-adc9-49e6-ad96-fab5b6f27cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b11c48638c0245779873bdadaec0f741, user_id=c1f8d61ec3e1426792000eed76c50a52, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:08.594849 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d0da1-adc9-49e6-ad96-fab5b6f27cbc admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:08.607556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1178/2338] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:25:08 2022] PUT /identity/v3/projects/b11c48638c0245779873bdadaec0f741/users/c1f8d61ec3e1426792000eed76c50a52/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:08.619732 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478c6f9-be78-4d17-9e93-ef5c74ffb3a4 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:08.620139 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478c6f9-be78-4d17-9e93-ef5c74ffb3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:08.620553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478c6f9-be78-4d17-9e93-ef5c74ffb3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:08.652362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-0478c6f9-be78-4d17-9e93-ef5c74ffb3a4 None None] MFA Rules not processed for user `c1f8d61ec3e1426792000eed76c50a52`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:08.653612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0478c6f9-be78-4d17-9e93-ef5c74ffb3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:08.727027 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1161/2339] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:08.738467 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d0da1-adc9-49e6-ad96-fab5b6f27cbc admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:08.740333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-af0d0da1-adc9-49e6-ad96-fab5b6f27cbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:08.762780 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:08.803240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d0da1-adc9-49e6-ad96-fab5b6f27cbc admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:08.807087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e389b-48a3-40b9-8fe9-c7ed65249687 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:08.807699 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e389b-48a3-40b9-8fe9-c7ed65249687 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:08.807981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e389b-48a3-40b9-8fe9-c7ed65249687 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:08.808253 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e389b-48a3-40b9-8fe9-c7ed65249687 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:08.865101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e389b-48a3-40b9-8fe9-c7ed65249687 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:08.866484 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e389b-48a3-40b9-8fe9-c7ed65249687 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:08.875398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1179/2340] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:25:18.725127 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=47 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:18.871497 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=39 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:20.960366 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478c6f9-be78-4d17-9e93-ef5c74ffb3a4 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:21.022515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478c6f9-be78-4d17-9e93-ef5c74ffb3a4 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:21.027049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758323f-6ef1-44be-8040-04d4918a8274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:21.027677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758323f-6ef1-44be-8040-04d4918a8274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:21.027960 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758323f-6ef1-44be-8040-04d4918a8274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:21.028238 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758323f-6ef1-44be-8040-04d4918a8274 admin admin] PATH_INFO: `/v3/users/a17f4c5306e749fea1d634be27287e25` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:21.029381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758323f-6ef1-44be-8040-04d4918a8274 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a17f4c5306e749fea1d634be27287e25)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:21.031283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758323f-6ef1-44be-8040-04d4918a8274 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:21.077623 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-0758323f-6ef1-44be-8040-04d4918a8274 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a17f4c5306e749fea1d634be27287e25'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:21.078261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-0758323f-6ef1-44be-8040-04d4918a8274 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a17f4c5306e749fea1d634be27287e25'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:21.090465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-0758323f-6ef1-44be-8040-04d4918a8274 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a17f4c5306e749fea1d634be27287e25'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:21.097830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-0758323f-6ef1-44be-8040-04d4918a8274 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a17f4c5306e749fea1d634be27287e25'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:21.098439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-0758323f-6ef1-44be-8040-04d4918a8274 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a17f4c5306e749fea1d634be27287e25'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:21.111013 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1162/2341] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:25:20 2022] DELETE /identity/v3/users/a17f4c5306e749fea1d634be27287e25 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:21.115607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40009b5-d8dd-4542-a85e-7a65373d5b55 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:21.116091 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40009b5-d8dd-4542-a85e-7a65373d5b55 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:21.116206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40009b5-d8dd-4542-a85e-7a65373d5b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:21.135415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-c40009b5-d8dd-4542-a85e-7a65373d5b55 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:21.136587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-c40009b5-d8dd-4542-a85e-7a65373d5b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:21.206997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1180/2342] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:21.218308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758323f-6ef1-44be-8040-04d4918a8274 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:21.220049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0758323f-6ef1-44be-8040-04d4918a8274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:21.298432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758323f-6ef1-44be-8040-04d4918a8274 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:21.302187 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563e22e-709d-4fe1-8dc1-f8118f78035a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:21.302836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563e22e-709d-4fe1-8dc1-f8118f78035a service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:21.303115 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563e22e-709d-4fe1-8dc1-f8118f78035a service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:21.303413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563e22e-709d-4fe1-8dc1-f8118f78035a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:21.313047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563e22e-709d-4fe1-8dc1-f8118f78035a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:21.314550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d563e22e-709d-4fe1-8dc1-f8118f78035a service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:21.324494 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1163/2343] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:25:21.681645 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40009b5-d8dd-4542-a85e-7a65373d5b55 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:21.683066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-c40009b5-d8dd-4542-a85e-7a65373d5b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:21.696819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40009b5-d8dd-4542-a85e-7a65373d5b55 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:21.701874 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f39860-75b1-44d7-a01e-0dc283063f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:21.702610 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f39860-75b1-44d7-a01e-0dc283063f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:21.702900 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f39860-75b1-44d7-a01e-0dc283063f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:21.703179 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f39860-75b1-44d7-a01e-0dc283063f94 admin admin] PATH_INFO: `/v3/projects/4ab2fa46e2234901b2eecb6884dff703` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:21.704380 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f39860-75b1-44d7-a01e-0dc283063f94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ab2fa46e2234901b2eecb6884dff703)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:21.706354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f39860-75b1-44d7-a01e-0dc283063f94 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:21.754504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-01f39860-75b1-44d7-a01e-0dc283063f94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ab2fa46e2234901b2eecb6884dff703', 'request_id': 'req-01f39860-75b1-44d7-a01e-0dc283063f94'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:21.761615 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-01f39860-75b1-44d7-a01e-0dc283063f94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ab2fa46e2234901b2eecb6884dff703', 'request_id': 'req-01f39860-75b1-44d7-a01e-0dc283063f94'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:21.768810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-01f39860-75b1-44d7-a01e-0dc283063f94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ab2fa46e2234901b2eecb6884dff703', 'request_id': 'req-01f39860-75b1-44d7-a01e-0dc283063f94'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:21.780430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-01f39860-75b1-44d7-a01e-0dc283063f94 admin admin] The token cache is being invalidate because project 4ab2fa46e2234901b2eecb6884dff703 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:25:21.780748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-01f39860-75b1-44d7-a01e-0dc283063f94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:21.781439 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-01f39860-75b1-44d7-a01e-0dc283063f94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:21.782924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1181/2344] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:25:21 2022] DELETE /identity/v3/projects/4ab2fa46e2234901b2eecb6884dff703 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:21.786952 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563e22e-709d-4fe1-8dc1-f8118f78035a service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:21.796856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563e22e-709d-4fe1-8dc1-f8118f78035a service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:21.816947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:21.871378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000da96c-7fa9-4d22-b609-53378e1412a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:21.871378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000da96c-7fa9-4d22-b609-53378e1412a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:21.871378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000da96c-7fa9-4d22-b609-53378e1412a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:21.871967 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000da96c-7fa9-4d22-b609-53378e1412a1 admin admin] PATH_INFO: `/v3/users/c1f8d61ec3e1426792000eed76c50a52` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:21.873863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000da96c-7fa9-4d22-b609-53378e1412a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1f8d61ec3e1426792000eed76c50a52)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:21.877749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000da96c-7fa9-4d22-b609-53378e1412a1 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:21.934287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-000da96c-7fa9-4d22-b609-53378e1412a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1f8d61ec3e1426792000eed76c50a52'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:21.935026 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-000da96c-7fa9-4d22-b609-53378e1412a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1f8d61ec3e1426792000eed76c50a52'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:21.949940 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-000da96c-7fa9-4d22-b609-53378e1412a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1f8d61ec3e1426792000eed76c50a52'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:21.961626 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-000da96c-7fa9-4d22-b609-53378e1412a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1f8d61ec3e1426792000eed76c50a52'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:21.962229 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-000da96c-7fa9-4d22-b609-53378e1412a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1f8d61ec3e1426792000eed76c50a52'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:21.975296 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1164/2345] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:25:21 2022] DELETE /identity/v3/users/c1f8d61ec3e1426792000eed76c50a52 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:22.271164 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f39860-75b1-44d7-a01e-0dc283063f94 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:22.272825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-01f39860-75b1-44d7-a01e-0dc283063f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:22.289394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f39860-75b1-44d7-a01e-0dc283063f94 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:22.361474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272f9cf-5678-4668-bdac-77fc63ef4298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:22.362117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272f9cf-5678-4668-bdac-77fc63ef4298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:22.362397 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272f9cf-5678-4668-bdac-77fc63ef4298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:22.362665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272f9cf-5678-4668-bdac-77fc63ef4298 admin admin] PATH_INFO: `/v3/projects/b11c48638c0245779873bdadaec0f741` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:22.363798 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272f9cf-5678-4668-bdac-77fc63ef4298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b11c48638c0245779873bdadaec0f741)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:22.366517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272f9cf-5678-4668-bdac-77fc63ef4298 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:22.407649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:22.436577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-1272f9cf-5678-4668-bdac-77fc63ef4298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b11c48638c0245779873bdadaec0f741', 'request_id': 'req-1272f9cf-5678-4668-bdac-77fc63ef4298'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:22.444855 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-1272f9cf-5678-4668-bdac-77fc63ef4298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b11c48638c0245779873bdadaec0f741', 'request_id': 'req-1272f9cf-5678-4668-bdac-77fc63ef4298'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:22.453926 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-1272f9cf-5678-4668-bdac-77fc63ef4298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b11c48638c0245779873bdadaec0f741', 'request_id': 'req-1272f9cf-5678-4668-bdac-77fc63ef4298'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:22.464753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-1272f9cf-5678-4668-bdac-77fc63ef4298 admin admin] The token cache is being invalidate because project b11c48638c0245779873bdadaec0f741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:25:22.465043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-1272f9cf-5678-4668-bdac-77fc63ef4298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:22.465724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-1272f9cf-5678-4668-bdac-77fc63ef4298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:22.467219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1182/2346] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:25:22 2022] DELETE /identity/v3/projects/b11c48638c0245779873bdadaec0f741 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:22.472414 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ef1c8-8427-4685-bb66-55c42b2f1b75 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:22.472854 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ef1c8-8427-4685-bb66-55c42b2f1b75 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:22.473160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ef1c8-8427-4685-bb66-55c42b2f1b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:22.494913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-ad8ef1c8-8427-4685-bb66-55c42b2f1b75 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:22.496125 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-ad8ef1c8-8427-4685-bb66-55c42b2f1b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:22.556856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1165/2347] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:22.564585 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cabf2-585a-45df-9d76-893312797b2f None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:22.564893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cabf2-585a-45df-9d76-893312797b2f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:22.565224 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730cabf2-585a-45df-9d76-893312797b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:22.585078 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-730cabf2-585a-45df-9d76-893312797b2f None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:22.586286 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-730cabf2-585a-45df-9d76-893312797b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:22.594041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1183/2348] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:22.598335 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ef1c8-8427-4685-bb66-55c42b2f1b75 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:22.608371 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ef1c8-8427-4685-bb66-55c42b2f1b75 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:22.612849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22213af-14db-40cf-bf4c-b824bdad5809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:22.613469 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22213af-14db-40cf-bf4c-b824bdad5809 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:22.613749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22213af-14db-40cf-bf4c-b824bdad5809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:22.614018 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22213af-14db-40cf-bf4c-b824bdad5809 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:22.614654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22213af-14db-40cf-bf4c-b824bdad5809 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:25:22.614980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22213af-14db-40cf-bf4c-b824bdad5809 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:22.616566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22213af-14db-40cf-bf4c-b824bdad5809 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:22.628031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1166/2349] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:25:22.632019 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730cabf2-585a-45df-9d76-893312797b2f None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:22.641181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730cabf2-585a-45df-9d76-893312797b2f None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:22.646002 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ab62c-8e92-4b1f-9708-88adfb044c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:22.646964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ab62c-8e92-4b1f-9708-88adfb044c4a admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:22.647388 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ab62c-8e92-4b1f-9708-88adfb044c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:22.647658 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ab62c-8e92-4b1f-9708-88adfb044c4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:22.648365 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ab62c-8e92-4b1f-9708-88adfb044c4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:22.650301 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ab62c-8e92-4b1f-9708-88adfb044c4a admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:22.678775 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1184/2350] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:25:22 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:22.682733 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22213af-14db-40cf-bf4c-b824bdad5809 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:22.693605 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22213af-14db-40cf-bf4c-b824bdad5809 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:22.725505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:22.763811 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf2d3f-6c5b-4ae4-9ff9-9ae128b71e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:22.764419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf2d3f-6c5b-4ae4-9ff9-9ae128b71e7f admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:22.764697 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf2d3f-6c5b-4ae4-9ff9-9ae128b71e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:22.764966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf2d3f-6c5b-4ae4-9ff9-9ae128b71e7f admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:22.765625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf2d3f-6c5b-4ae4-9ff9-9ae128b71e7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:22.767516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf2d3f-6c5b-4ae4-9ff9-9ae128b71e7f admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:22.792224 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1167/2351] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:25:22 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:22.796223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ab62c-8e92-4b1f-9708-88adfb044c4a admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:22.806659 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ab62c-8e92-4b1f-9708-88adfb044c4a admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:22.811224 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2ca42-7f7f-4029-a299-1a0f1ef14d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:22.811813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2ca42-7f7f-4029-a299-1a0f1ef14d3f admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:22.812087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2ca42-7f7f-4029-a299-1a0f1ef14d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:22.812354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2ca42-7f7f-4029-a299-1a0f1ef14d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:22.812985 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2ca42-7f7f-4029-a299-1a0f1ef14d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:22.814598 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2ca42-7f7f-4029-a299-1a0f1ef14d3f admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:22.825110 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1185/2352] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:22.828950 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf2d3f-6c5b-4ae4-9ff9-9ae128b71e7f admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:22.839573 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf2d3f-6c5b-4ae4-9ff9-9ae128b71e7f admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:22.844274 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c133856-954c-48cb-bb27-b5790f22623a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:22.844868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c133856-954c-48cb-bb27-b5790f22623a admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:22.845179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c133856-954c-48cb-bb27-b5790f22623a admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:22.845461 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c133856-954c-48cb-bb27-b5790f22623a admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:22.846098 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c133856-954c-48cb-bb27-b5790f22623a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:22.847955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c133856-954c-48cb-bb27-b5790f22623a admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:22.859594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1168/2353] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:22.863577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2ca42-7f7f-4029-a299-1a0f1ef14d3f admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:22.873497 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2ca42-7f7f-4029-a299-1a0f1ef14d3f admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:22.878129 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fdceff-6dfe-4c60-b636-925728ca4044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:22.878819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdceff-6dfe-4c60-b636-925728ca4044 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:22.879098 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdceff-6dfe-4c60-b636-925728ca4044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:22.879373 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fdceff-6dfe-4c60-b636-925728ca4044 admin admin] PATH_INFO: `/v3/projects/937511abf270462ba94a4b56e0594b51/users/ec3dc4e21d564618afb548369cd214c1/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:22.894945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fdceff-6dfe-4c60-b636-925728ca4044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=937511abf270462ba94a4b56e0594b51, user_id=ec3dc4e21d564618afb548369cd214c1, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:22.898464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fdceff-6dfe-4c60-b636-925728ca4044 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:22.911461 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1186/2354] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:25:22 2022] PUT /identity/v3/projects/937511abf270462ba94a4b56e0594b51/users/ec3dc4e21d564618afb548369cd214c1/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:22.983020 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e41c9d-32ac-4ae9-8a67-99b2ce9ff1aa None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:22.983514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e41c9d-32ac-4ae9-8a67-99b2ce9ff1aa None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:22.983611 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e41c9d-32ac-4ae9-8a67-99b2ce9ff1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:23.013106 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-20e41c9d-32ac-4ae9-8a67-99b2ce9ff1aa None None] MFA Rules not processed for user `ec3dc4e21d564618afb548369cd214c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:23.014399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-20e41c9d-32ac-4ae9-8a67-99b2ce9ff1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:23.081770 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1169/2355] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:23.086974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fdceff-6dfe-4c60-b636-925728ca4044 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:23.150931 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fdceff-6dfe-4c60-b636-925728ca4044 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:23.155733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ccfdc-a0f2-4fb4-8ca6-aaa9b4a0e8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:23.156344 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ccfdc-a0f2-4fb4-8ca6-aaa9b4a0e8d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:23.156624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ccfdc-a0f2-4fb4-8ca6-aaa9b4a0e8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:23.156925 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ccfdc-a0f2-4fb4-8ca6-aaa9b4a0e8d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:23.157781 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ccfdc-a0f2-4fb4-8ca6-aaa9b4a0e8d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:23.159727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ccfdc-a0f2-4fb4-8ca6-aaa9b4a0e8d5 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:23.193395 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1187/2356] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:25:23 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:23.194759 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e41c9d-32ac-4ae9-8a67-99b2ce9ff1aa None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:23.205812 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e41c9d-32ac-4ae9-8a67-99b2ce9ff1aa None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:23.263660 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1d138-f76c-4524-be92-359834e099cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:23.264189 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1d138-f76c-4524-be92-359834e099cf admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:23.264466 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1d138-f76c-4524-be92-359834e099cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:23.264734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1d138-f76c-4524-be92-359834e099cf admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:23.265383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1d138-f76c-4524-be92-359834e099cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:23.267268 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1d138-f76c-4524-be92-359834e099cf admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:23.288257 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1170/2357] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:25:23 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:23.292234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ccfdc-a0f2-4fb4-8ca6-aaa9b4a0e8d5 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:23.303940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ccfdc-a0f2-4fb4-8ca6-aaa9b4a0e8d5 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:23.308583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6fd4e-9078-46b4-9269-2e06f09f577c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:23.309217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6fd4e-9078-46b4-9269-2e06f09f577c admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:23.309523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6fd4e-9078-46b4-9269-2e06f09f577c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:23.309825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6fd4e-9078-46b4-9269-2e06f09f577c admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:23.310454 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6fd4e-9078-46b4-9269-2e06f09f577c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:23.312035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6fd4e-9078-46b4-9269-2e06f09f577c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:23.325553 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1188/2358] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:23.329715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1d138-f76c-4524-be92-359834e099cf admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:23.340426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1d138-f76c-4524-be92-359834e099cf admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:23.345123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7152232-fada-4db6-977d-2d516f4c202d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:23.345751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7152232-fada-4db6-977d-2d516f4c202d admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:23.346043 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7152232-fada-4db6-977d-2d516f4c202d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:23.346301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7152232-fada-4db6-977d-2d516f4c202d admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:23.346935 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7152232-fada-4db6-977d-2d516f4c202d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:23.348776 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7152232-fada-4db6-977d-2d516f4c202d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:23.359138 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1171/2359] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:23.363146 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6fd4e-9078-46b4-9269-2e06f09f577c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:23.373530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6fd4e-9078-46b4-9269-2e06f09f577c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:23.378026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159fb6e-799f-447b-8665-8669090dc9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:23.378657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159fb6e-799f-447b-8665-8669090dc9eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:23.378933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159fb6e-799f-447b-8665-8669090dc9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:23.379202 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159fb6e-799f-447b-8665-8669090dc9eb admin admin] PATH_INFO: `/v3/projects/72710bb57e4842fbbfa46abad4353e85/users/50926e9eb0de4cb5bdd887cbb3ad34cd/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:23.393850 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159fb6e-799f-447b-8665-8669090dc9eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=72710bb57e4842fbbfa46abad4353e85, user_id=50926e9eb0de4cb5bdd887cbb3ad34cd, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:23.396446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159fb6e-799f-447b-8665-8669090dc9eb admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:23.404902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:23.409905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1189/2360] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:25:23 2022] PUT /identity/v3/projects/72710bb57e4842fbbfa46abad4353e85/users/50926e9eb0de4cb5bdd887cbb3ad34cd/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:23.420395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4617c1-8f92-496e-bd89-dacbf7205556 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:23.420875 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4617c1-8f92-496e-bd89-dacbf7205556 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:23.421296 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4617c1-8f92-496e-bd89-dacbf7205556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:23.450494 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-ad4617c1-8f92-496e-bd89-dacbf7205556 None None] MFA Rules not processed for user `50926e9eb0de4cb5bdd887cbb3ad34cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:23.451645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-ad4617c1-8f92-496e-bd89-dacbf7205556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:23.455632 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:23.467708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159fb6e-799f-447b-8665-8669090dc9eb admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:23.469404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-c159fb6e-799f-447b-8665-8669090dc9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:23.525971 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1172/2361] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:23.530258 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4617c1-8f92-496e-bd89-dacbf7205556 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:23.537399 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159fb6e-799f-447b-8665-8669090dc9eb admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:23.540920 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4617c1-8f92-496e-bd89-dacbf7205556 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:23.542360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85f6f0-e930-4835-9bda-efec923675dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:23.542996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85f6f0-e930-4835-9bda-efec923675dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:23.543277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85f6f0-e930-4835-9bda-efec923675dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:23.543677 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85f6f0-e930-4835-9bda-efec923675dd admin admin] PATH_INFO: `/v3/users/564bc620e923434b9389923314499d6a` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:23.544813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85f6f0-e930-4835-9bda-efec923675dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=564bc620e923434b9389923314499d6a)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:23.545627 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8091d-d1c8-4080-9910-9bda9b32491a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:23.546482 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8091d-d1c8-4080-9910-9bda9b32491a admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:23.546482 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8091d-d1c8-4080-9910-9bda9b32491a admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:23.546749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e8091d-d1c8-4080-9910-9bda9b32491a admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:23.547071 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85f6f0-e930-4835-9bda-efec923675dd admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:23.547371 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8091d-d1c8-4080-9910-9bda9b32491a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:23.549388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e8091d-d1c8-4080-9910-9bda9b32491a admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:23.583345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1173/2362] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:25:23 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:25:23.587303 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8091d-d1c8-4080-9910-9bda9b32491a admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:23.606505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-1e85f6f0-e930-4835-9bda-efec923675dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '564bc620e923434b9389923314499d6a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:23.607167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-1e85f6f0-e930-4835-9bda-efec923675dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '564bc620e923434b9389923314499d6a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:23.615526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-1e85f6f0-e930-4835-9bda-efec923675dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '564bc620e923434b9389923314499d6a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:23.616343 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-1e85f6f0-e930-4835-9bda-efec923675dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '564bc620e923434b9389923314499d6a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:23.623526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-1e85f6f0-e930-4835-9bda-efec923675dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '564bc620e923434b9389923314499d6a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:23.642072 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1190/2363] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:25:23 2022] DELETE /identity/v3/users/564bc620e923434b9389923314499d6a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:25:23.646650 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bed2d-118c-43b9-b9fb-b5efef7d3090 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:23.647019 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bed2d-118c-43b9-b9fb-b5efef7d3090 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:23.647296 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bed2d-118c-43b9-b9fb-b5efef7d3090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:23.654471 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e8091d-d1c8-4080-9910-9bda9b32491a admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:23.666203 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-b21bed2d-118c-43b9-b9fb-b5efef7d3090 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:23.667391 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b21bed2d-118c-43b9-b9fb-b5efef7d3090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:23.709912 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461715ff-1635-46c6-b224-e5161a28b9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:23.710661 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461715ff-1635-46c6-b224-e5161a28b9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:23.710940 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461715ff-1635-46c6-b224-e5161a28b9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:23.711208 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461715ff-1635-46c6-b224-e5161a28b9a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:23.711827 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461715ff-1635-46c6-b224-e5161a28b9a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:23.713913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461715ff-1635-46c6-b224-e5161a28b9a4 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:23.727250 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1191/2364] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:23.737522 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bed2d-118c-43b9-b9fb-b5efef7d3090 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:23.739017 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b21bed2d-118c-43b9-b9fb-b5efef7d3090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:23.745239 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1174/2365] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:25:23 2022] POST /identity/v3/users => generated 298 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:25:23.749316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461715ff-1635-46c6-b224-e5161a28b9a4 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:23.751781 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-461715ff-1635-46c6-b224-e5161a28b9a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:23.766608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461715ff-1635-46c6-b224-e5161a28b9a4 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:23.772099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c014a8-568d-409d-aea3-047654f2503e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:23.772891 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c014a8-568d-409d-aea3-047654f2503e admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:23.773112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c014a8-568d-409d-aea3-047654f2503e admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:23.773416 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c014a8-568d-409d-aea3-047654f2503e admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:23.774058 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c014a8-568d-409d-aea3-047654f2503e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:23.775548 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:23.776413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c014a8-568d-409d-aea3-047654f2503e admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:23.797160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1175/2366] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:23.798373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c014a8-568d-409d-aea3-047654f2503e admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:23.834356 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c014a8-568d-409d-aea3-047654f2503e admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:23.839458 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0e243-3737-414e-b733-946528e4d595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:23.841054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0e243-3737-414e-b733-946528e4d595 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:23.841443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bed2d-118c-43b9-b9fb-b5efef7d3090 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:23.841730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0e243-3737-414e-b733-946528e4d595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:23.842223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0e243-3737-414e-b733-946528e4d595 admin admin] PATH_INFO: `/v3/projects/724c780ea76f4d0cbdb787a17e565468/users/e11003b9c9414e17bc9cc03f949abe20/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:23.846729 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:23.847993 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744ad4f-2a44-41b6-91b1-c176977d9978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:23.848660 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744ad4f-2a44-41b6-91b1-c176977d9978 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:23.849095 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744ad4f-2a44-41b6-91b1-c176977d9978 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:23.850024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744ad4f-2a44-41b6-91b1-c176977d9978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:23.863356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744ad4f-2a44-41b6-91b1-c176977d9978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:23.866126 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744ad4f-2a44-41b6-91b1-c176977d9978 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:23.877950 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1192/2367] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:25:23.882561 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0e243-3737-414e-b733-946528e4d595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=724c780ea76f4d0cbdb787a17e565468, user_id=e11003b9c9414e17bc9cc03f949abe20, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:23.886312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0e243-3737-414e-b733-946528e4d595 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:23.908123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1176/2368] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:25:23 2022] PUT /identity/v3/projects/724c780ea76f4d0cbdb787a17e565468/users/e11003b9c9414e17bc9cc03f949abe20/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:23.913315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744ad4f-2a44-41b6-91b1-c176977d9978 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:23.931874 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744ad4f-2a44-41b6-91b1-c176977d9978 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:24.057742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2ca28-3ea7-4155-92cc-a36cb825b2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:24.058140 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2ca28-3ea7-4155-92cc-a36cb825b2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:24.058430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2ca28-3ea7-4155-92cc-a36cb825b2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:24.058702 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2ca28-3ea7-4155-92cc-a36cb825b2b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:24.059416 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2ca28-3ea7-4155-92cc-a36cb825b2b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:24.062622 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2ca28-3ea7-4155-92cc-a36cb825b2b3 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:24.091827 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1193/2369] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:24.095358 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0e243-3737-414e-b733-946528e4d595 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:24.110093 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0e243-3737-414e-b733-946528e4d595 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:24.129155 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff057a-ddd8-4cbc-aa2b-03a78f82302e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:24.129965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff057a-ddd8-4cbc-aa2b-03a78f82302e admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:24.130335 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff057a-ddd8-4cbc-aa2b-03a78f82302e admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:24.131472 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff057a-ddd8-4cbc-aa2b-03a78f82302e admin admin] PATH_INFO: `/v3/projects/724c780ea76f4d0cbdb787a17e565468/users/e11003b9c9414e17bc9cc03f949abe20/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:24.135316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff057a-ddd8-4cbc-aa2b-03a78f82302e admin admin] RBAC: Authorizing `identity:create_grant(project_id=724c780ea76f4d0cbdb787a17e565468, user_id=e11003b9c9414e17bc9cc03f949abe20, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:24.142923 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff057a-ddd8-4cbc-aa2b-03a78f82302e admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:24.163774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1177/2370] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:25:24 2022] PUT /identity/v3/projects/724c780ea76f4d0cbdb787a17e565468/users/e11003b9c9414e17bc9cc03f949abe20/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:24.187142 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2478a00-d9fb-4090-a89b-632be2a19791 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:24.187430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2478a00-d9fb-4090-a89b-632be2a19791 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:24.187858 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2478a00-d9fb-4090-a89b-632be2a19791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:24.248151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-f2478a00-d9fb-4090-a89b-632be2a19791 None None] MFA Rules not processed for user `e11003b9c9414e17bc9cc03f949abe20`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:24.250732 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-f2478a00-d9fb-4090-a89b-632be2a19791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:24.355555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1194/2371] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:24.365123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff057a-ddd8-4cbc-aa2b-03a78f82302e admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:24.367517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-fdff057a-ddd8-4cbc-aa2b-03a78f82302e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:24.389531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2478a00-d9fb-4090-a89b-632be2a19791 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:24.391053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-f2478a00-d9fb-4090-a89b-632be2a19791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:24.450388 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:24.460944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff057a-ddd8-4cbc-aa2b-03a78f82302e admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:24.464539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e181e-9dbb-4291-ac62-b147db5ac4e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:24.465171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e181e-9dbb-4291-ac62-b147db5ac4e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:24.465448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e181e-9dbb-4291-ac62-b147db5ac4e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:24.465723 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e181e-9dbb-4291-ac62-b147db5ac4e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:24.467459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-012e181e-9dbb-4291-ac62-b147db5ac4e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:24.470158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2478a00-d9fb-4090-a89b-632be2a19791 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:24.475207 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d8af1-ca39-4eb1-b778-7ea9cfdab792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:24.475837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d8af1-ca39-4eb1-b778-7ea9cfdab792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:24.476118 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d8af1-ca39-4eb1-b778-7ea9cfdab792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:24.476384 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d8af1-ca39-4eb1-b778-7ea9cfdab792 admin admin] PATH_INFO: `/v3/projects/a5fc5cc3087f4ba68bba4daa437b7cc6` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:24.477505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d8af1-ca39-4eb1-b778-7ea9cfdab792 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5fc5cc3087f4ba68bba4daa437b7cc6)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:24.479417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d8af1-ca39-4eb1-b778-7ea9cfdab792 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:24.527441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6e5d8af1-ca39-4eb1-b778-7ea9cfdab792 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5fc5cc3087f4ba68bba4daa437b7cc6', 'request_id': 'req-6e5d8af1-ca39-4eb1-b778-7ea9cfdab792'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:24.528838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e181e-9dbb-4291-ac62-b147db5ac4e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:24.530404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e181e-9dbb-4291-ac62-b147db5ac4e9 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:24.533769 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6e5d8af1-ca39-4eb1-b778-7ea9cfdab792 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5fc5cc3087f4ba68bba4daa437b7cc6', 'request_id': 'req-6e5d8af1-ca39-4eb1-b778-7ea9cfdab792'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:24.539623 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1178/2372] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:25:24.543018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6e5d8af1-ca39-4eb1-b778-7ea9cfdab792 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5fc5cc3087f4ba68bba4daa437b7cc6', 'request_id': 'req-6e5d8af1-ca39-4eb1-b778-7ea9cfdab792'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:24.554137 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6e5d8af1-ca39-4eb1-b778-7ea9cfdab792 admin admin] The token cache is being invalidate because project a5fc5cc3087f4ba68bba4daa437b7cc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:25:24.554435 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6e5d8af1-ca39-4eb1-b778-7ea9cfdab792 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:24.555097 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-6e5d8af1-ca39-4eb1-b778-7ea9cfdab792 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:24.556615 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1195/2373] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:25:24 2022] DELETE /identity/v3/projects/a5fc5cc3087f4ba68bba4daa437b7cc6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:24.561584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896f442-09a3-499b-bbbe-e8bf03bfc0bf None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:24.561869 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896f442-09a3-499b-bbbe-e8bf03bfc0bf None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:24.562143 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896f442-09a3-499b-bbbe-e8bf03bfc0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:24.582675 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-3896f442-09a3-499b-bbbe-e8bf03bfc0bf None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:24.584599 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-3896f442-09a3-499b-bbbe-e8bf03bfc0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:24.648293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1179/2374] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:24.657765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513d2fe-419f-4a68-8267-ad09363efb83 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:24.658414 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513d2fe-419f-4a68-8267-ad09363efb83 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:24.658798 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513d2fe-419f-4a68-8267-ad09363efb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:24.692137 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-3513d2fe-419f-4a68-8267-ad09363efb83 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:24.694139 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-3513d2fe-419f-4a68-8267-ad09363efb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:24.703506 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1196/2375] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:24.708805 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896f442-09a3-499b-bbbe-e8bf03bfc0bf None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:24.723762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896f442-09a3-499b-bbbe-e8bf03bfc0bf None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:24.730173 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077096bc-7c95-4059-9420-cac5e107bff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:24.730959 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077096bc-7c95-4059-9420-cac5e107bff3 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:24.731308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077096bc-7c95-4059-9420-cac5e107bff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:24.731620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077096bc-7c95-4059-9420-cac5e107bff3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:24.732484 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077096bc-7c95-4059-9420-cac5e107bff3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:25:24.732916 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077096bc-7c95-4059-9420-cac5e107bff3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:24.734762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077096bc-7c95-4059-9420-cac5e107bff3 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:24.751048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1180/2376] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:25:24.755516 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513d2fe-419f-4a68-8267-ad09363efb83 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:24.765066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513d2fe-419f-4a68-8267-ad09363efb83 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:24.770399 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247001d-6022-4881-a339-bfc627d3d39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:24.771627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247001d-6022-4881-a339-bfc627d3d39a admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:24.771983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247001d-6022-4881-a339-bfc627d3d39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:24.772339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247001d-6022-4881-a339-bfc627d3d39a admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:24.773051 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247001d-6022-4881-a339-bfc627d3d39a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:24.775075 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247001d-6022-4881-a339-bfc627d3d39a admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:24.813464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1197/2377] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:25:24 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:24.817874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077096bc-7c95-4059-9420-cac5e107bff3 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:24.834201 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077096bc-7c95-4059-9420-cac5e107bff3 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:24.852899 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:24.928129 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e265d81-a533-4cd2-b3f2-48c6326c68ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:24.929157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e265d81-a533-4cd2-b3f2-48c6326c68ac admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:24.929731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e265d81-a533-4cd2-b3f2-48c6326c68ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:24.930215 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e265d81-a533-4cd2-b3f2-48c6326c68ac admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:24.931112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e265d81-a533-4cd2-b3f2-48c6326c68ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:24.933636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e265d81-a533-4cd2-b3f2-48c6326c68ac admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:24.972381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1181/2378] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:25:24 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:24.973846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247001d-6022-4881-a339-bfc627d3d39a admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:24.991477 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247001d-6022-4881-a339-bfc627d3d39a admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:25.000990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6346d3b-adfb-4ce7-a2f8-9ef481a2da3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:25.001716 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6346d3b-adfb-4ce7-a2f8-9ef481a2da3b admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:25.001944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6346d3b-adfb-4ce7-a2f8-9ef481a2da3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:25.002219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6346d3b-adfb-4ce7-a2f8-9ef481a2da3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:25.002874 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6346d3b-adfb-4ce7-a2f8-9ef481a2da3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:25.009860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6346d3b-adfb-4ce7-a2f8-9ef481a2da3b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:25.040712 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1198/2379] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:25:25.044848 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e265d81-a533-4cd2-b3f2-48c6326c68ac admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:25.061982 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e265d81-a533-4cd2-b3f2-48c6326c68ac admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:25.074608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f49918-d0ae-48de-998a-d6185947ad7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:25.075242 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f49918-d0ae-48de-998a-d6185947ad7d admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:25.075522 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f49918-d0ae-48de-998a-d6185947ad7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:25.075790 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f49918-d0ae-48de-998a-d6185947ad7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:25.076496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f49918-d0ae-48de-998a-d6185947ad7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:25.078200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f49918-d0ae-48de-998a-d6185947ad7d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:25.091127 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1182/2380] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:25.095730 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6346d3b-adfb-4ce7-a2f8-9ef481a2da3b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:25.107922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6346d3b-adfb-4ce7-a2f8-9ef481a2da3b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:25.113149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9addaa54-c7cb-4d6d-bbd0-ee3fadb51926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:25.113941 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addaa54-c7cb-4d6d-bbd0-ee3fadb51926 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:25.114336 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addaa54-c7cb-4d6d-bbd0-ee3fadb51926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:25.114734 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addaa54-c7cb-4d6d-bbd0-ee3fadb51926 admin admin] PATH_INFO: `/v3/projects/14944124ab8a4487b879592a02981d12/users/d12d46f42d73486fbbc3a1c2df0f3ab9/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:25.129682 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9addaa54-c7cb-4d6d-bbd0-ee3fadb51926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14944124ab8a4487b879592a02981d12, user_id=d12d46f42d73486fbbc3a1c2df0f3ab9, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:25.132817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9addaa54-c7cb-4d6d-bbd0-ee3fadb51926 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:25.145269 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1199/2381] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:25:25 2022] PUT /identity/v3/projects/14944124ab8a4487b879592a02981d12/users/d12d46f42d73486fbbc3a1c2df0f3ab9/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:25.157080 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3622ed5-f3dc-4c11-9149-b0a3a7a9183d None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:25.157544 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3622ed5-f3dc-4c11-9149-b0a3a7a9183d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:25.157638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3622ed5-f3dc-4c11-9149-b0a3a7a9183d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:25.213184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-c3622ed5-f3dc-4c11-9149-b0a3a7a9183d None None] MFA Rules not processed for user `d12d46f42d73486fbbc3a1c2df0f3ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:25.215580 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c3622ed5-f3dc-4c11-9149-b0a3a7a9183d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:25.306508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1183/2382] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:25.319175 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9addaa54-c7cb-4d6d-bbd0-ee3fadb51926 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:25.320979 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9addaa54-c7cb-4d6d-bbd0-ee3fadb51926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:25.407595 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738070c2-cade-404a-bd05-2098d424323f None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:25.407887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738070c2-cade-404a-bd05-2098d424323f None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:25.408152 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738070c2-cade-404a-bd05-2098d424323f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:25.412257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9addaa54-c7cb-4d6d-bbd0-ee3fadb51926 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:25.417374 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d61a8b-cffb-4f86-8b30-e6a40e1ee5fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:25.418191 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d61a8b-cffb-4f86-8b30-e6a40e1ee5fc service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:25.418733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d61a8b-cffb-4f86-8b30-e6a40e1ee5fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:25.419107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d61a8b-cffb-4f86-8b30-e6a40e1ee5fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:25.431300 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d61a8b-cffb-4f86-8b30-e6a40e1ee5fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:25.432864 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d61a8b-cffb-4f86-8b30-e6a40e1ee5fc service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:25.457343 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1200/2383] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:25:25.479587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-738070c2-cade-404a-bd05-2098d424323f None None] MFA Rules not processed for user `bc69cf267bea41bf890bc7551d84ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:25.482843 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-738070c2-cade-404a-bd05-2098d424323f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:25.558832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1184/2384] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 12:25:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 28 12:25:25.573285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d61a8b-cffb-4f86-8b30-e6a40e1ee5fc service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:25.575668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-95d61a8b-cffb-4f86-8b30-e6a40e1ee5fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:25.603634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:25.681678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d61a8b-cffb-4f86-8b30-e6a40e1ee5fc service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:25.687458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89e96c-ea36-4061-bed6-541f02040b27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975a8d7349994520bcbfb5f75e52aac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:25.688329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89e96c-ea36-4061-bed6-541f02040b27 service swift] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:25.688710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89e96c-ea36-4061-bed6-541f02040b27 service swift] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:25.689078 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89e96c-ea36-4061-bed6-541f02040b27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:25.699468 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89e96c-ea36-4061-bed6-541f02040b27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:25.701539 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89e96c-ea36-4061-bed6-541f02040b27 service swift] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:25.725978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1201/2385] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 12:25:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 12:25:27.281775 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738070c2-cade-404a-bd05-2098d424323f None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:27.291976 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738070c2-cade-404a-bd05-2098d424323f None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:27.295740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830598b-9188-4158-9f55-7922754841be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:27.296374 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830598b-9188-4158-9f55-7922754841be service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:27.296626 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830598b-9188-4158-9f55-7922754841be service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:27.296908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830598b-9188-4158-9f55-7922754841be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:27.298678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1830598b-9188-4158-9f55-7922754841be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:27.325903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:27.367342 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830598b-9188-4158-9f55-7922754841be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:27.368691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830598b-9188-4158-9f55-7922754841be service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:27.378296 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1185/2386] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:25:28.045811 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10364fce-4459-47d4-a32f-61e4d8ae6902 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:28.046325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10364fce-4459-47d4-a32f-61e4d8ae6902 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:28.046325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10364fce-4459-47d4-a32f-61e4d8ae6902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:28.072488 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-10364fce-4459-47d4-a32f-61e4d8ae6902 None None] MFA Rules not processed for user `d12d46f42d73486fbbc3a1c2df0f3ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:28.073757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-10364fce-4459-47d4-a32f-61e4d8ae6902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:28.085066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1202/2387] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:28.201421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830598b-9188-4158-9f55-7922754841be service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:28.217376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830598b-9188-4158-9f55-7922754841be service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:28.222458 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c7d77-15aa-4963-8bdc-7fbba3deabbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:28.223161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c7d77-15aa-4963-8bdc-7fbba3deabbb service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:28.224159 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c7d77-15aa-4963-8bdc-7fbba3deabbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:28.224655 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c7d77-15aa-4963-8bdc-7fbba3deabbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:28.226794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-6b6c7d77-15aa-4963-8bdc-7fbba3deabbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:28.312882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c7d77-15aa-4963-8bdc-7fbba3deabbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:28.314542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c7d77-15aa-4963-8bdc-7fbba3deabbb service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:28.326885 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1186/2388] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:25:31.899919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10364fce-4459-47d4-a32f-61e4d8ae6902 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:31.901614 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-10364fce-4459-47d4-a32f-61e4d8ae6902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:31.971301 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10364fce-4459-47d4-a32f-61e4d8ae6902 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:31.977850 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3e13e-583f-405f-ae85-785d8de886f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:31.978504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3e13e-583f-405f-ae85-785d8de886f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:31.978780 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3e13e-583f-405f-ae85-785d8de886f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:31.979049 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3e13e-583f-405f-ae85-785d8de886f0 admin admin] PATH_INFO: `/v3/users/17d7cab966f64b479831b4c57897f20a` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:31.980187 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3e13e-583f-405f-ae85-785d8de886f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17d7cab966f64b479831b4c57897f20a)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:31.982146 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3e13e-583f-405f-ae85-785d8de886f0 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:32.049291 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-a6b3e13e-583f-405f-ae85-785d8de886f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17d7cab966f64b479831b4c57897f20a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:32.050274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-a6b3e13e-583f-405f-ae85-785d8de886f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17d7cab966f64b479831b4c57897f20a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:32.058556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-a6b3e13e-583f-405f-ae85-785d8de886f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17d7cab966f64b479831b4c57897f20a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:32.059277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-a6b3e13e-583f-405f-ae85-785d8de886f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17d7cab966f64b479831b4c57897f20a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:32.069148 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-a6b3e13e-583f-405f-ae85-785d8de886f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17d7cab966f64b479831b4c57897f20a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:32.092998 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1203/2389] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:25:31 2022] DELETE /identity/v3/users/17d7cab966f64b479831b4c57897f20a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:32.094232 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e26df7-18de-48a2-b5be-bb8bb7a8faf5 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:32.094741 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e26df7-18de-48a2-b5be-bb8bb7a8faf5 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:32.094831 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e26df7-18de-48a2-b5be-bb8bb7a8faf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:32.117264 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-65e26df7-18de-48a2-b5be-bb8bb7a8faf5 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:32.119110 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-65e26df7-18de-48a2-b5be-bb8bb7a8faf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:32.238114 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3e13e-583f-405f-ae85-785d8de886f0 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:32.241557 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a6b3e13e-583f-405f-ae85-785d8de886f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:32.245565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1187/2390] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:25:32.293782 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:32.395200 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3e13e-583f-405f-ae85-785d8de886f0 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:32.399086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60413f68-f5d4-48e0-b879-2d0dd4e9c4dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:32.399720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60413f68-f5d4-48e0-b879-2d0dd4e9c4dc service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:32.400006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60413f68-f5d4-48e0-b879-2d0dd4e9c4dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:32.400286 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60413f68-f5d4-48e0-b879-2d0dd4e9c4dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:32.438850 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60413f68-f5d4-48e0-b879-2d0dd4e9c4dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:32.440940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60413f68-f5d4-48e0-b879-2d0dd4e9c4dc service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:32.469549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1204/2391] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:25:32.820367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e26df7-18de-48a2-b5be-bb8bb7a8faf5 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:32.822490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-65e26df7-18de-48a2-b5be-bb8bb7a8faf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:32.840080 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e26df7-18de-48a2-b5be-bb8bb7a8faf5 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:32.846303 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439d197-7c3f-4358-a204-a70d1c4175e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:32.847429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439d197-7c3f-4358-a204-a70d1c4175e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:32.847962 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439d197-7c3f-4358-a204-a70d1c4175e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:32.848453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439d197-7c3f-4358-a204-a70d1c4175e5 admin admin] PATH_INFO: `/v3/projects/4b2020bddfbe4145936e64db6c1c8a06` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:32.858195 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439d197-7c3f-4358-a204-a70d1c4175e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b2020bddfbe4145936e64db6c1c8a06)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:32.861278 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439d197-7c3f-4358-a204-a70d1c4175e5 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:32.971593 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-4439d197-7c3f-4358-a204-a70d1c4175e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b2020bddfbe4145936e64db6c1c8a06', 'request_id': 'req-4439d197-7c3f-4358-a204-a70d1c4175e5'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:32.987229 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-4439d197-7c3f-4358-a204-a70d1c4175e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b2020bddfbe4145936e64db6c1c8a06', 'request_id': 'req-4439d197-7c3f-4358-a204-a70d1c4175e5'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:32.993034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:33.010328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-4439d197-7c3f-4358-a204-a70d1c4175e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b2020bddfbe4145936e64db6c1c8a06', 'request_id': 'req-4439d197-7c3f-4358-a204-a70d1c4175e5'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:33.055456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-4439d197-7c3f-4358-a204-a70d1c4175e5 admin admin] The token cache is being invalidate because project 4b2020bddfbe4145936e64db6c1c8a06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:25:33.055761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-4439d197-7c3f-4358-a204-a70d1c4175e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:33.056424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-4439d197-7c3f-4358-a204-a70d1c4175e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:33.058289 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1188/2392] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:25:32 2022] DELETE /identity/v3/projects/4b2020bddfbe4145936e64db6c1c8a06 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:33.063153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa14b7-c822-48f7-a904-1e759d118067 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:33.063703 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa14b7-c822-48f7-a904-1e759d118067 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:33.063703 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa14b7-c822-48f7-a904-1e759d118067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:33.093179 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-1daa14b7-c822-48f7-a904-1e759d118067 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:33.094636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-1daa14b7-c822-48f7-a904-1e759d118067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:33.214692 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1205/2393] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:33.222631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5d785-a4ad-4d23-bbe1-babbd906a1fd None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:33.223183 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5d785-a4ad-4d23-bbe1-babbd906a1fd None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:33.223183 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5d785-a4ad-4d23-bbe1-babbd906a1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:33.265589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-36e5d785-a4ad-4d23-bbe1-babbd906a1fd None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:33.266810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-36e5d785-a4ad-4d23-bbe1-babbd906a1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:33.275543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1189/2394] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:33.280087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa14b7-c822-48f7-a904-1e759d118067 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:33.290590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa14b7-c822-48f7-a904-1e759d118067 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:33.295212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4e527-0229-4f26-9e5b-d20d9051ebb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:33.295827 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4e527-0229-4f26-9e5b-d20d9051ebb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:33.296546 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4e527-0229-4f26-9e5b-d20d9051ebb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:33.296825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4e527-0229-4f26-9e5b-d20d9051ebb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:33.297538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4e527-0229-4f26-9e5b-d20d9051ebb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:25:33.297867 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4e527-0229-4f26-9e5b-d20d9051ebb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:33.299437 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4e527-0229-4f26-9e5b-d20d9051ebb9 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:33.315763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1206/2395] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:25:33.317083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5d785-a4ad-4d23-bbe1-babbd906a1fd None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:33.327194 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5d785-a4ad-4d23-bbe1-babbd906a1fd None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:33.332013 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d629e1-eada-40f1-a83d-ebf03d072b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:33.332654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d629e1-eada-40f1-a83d-ebf03d072b88 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:33.332940 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d629e1-eada-40f1-a83d-ebf03d072b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:33.333942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d629e1-eada-40f1-a83d-ebf03d072b88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:33.334410 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d629e1-eada-40f1-a83d-ebf03d072b88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:33.336319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d629e1-eada-40f1-a83d-ebf03d072b88 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:33.369054 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1190/2396] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:25:33 2022] POST /identity/v3/projects => generated 361 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:25:33.370240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4e527-0229-4f26-9e5b-d20d9051ebb9 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:33.381391 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4e527-0229-4f26-9e5b-d20d9051ebb9 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:33.440995 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbbd938-19d8-49a2-a50e-1f18a7e4f194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:33.441892 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbd938-19d8-49a2-a50e-1f18a7e4f194 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:33.442248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbd938-19d8-49a2-a50e-1f18a7e4f194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:33.443151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbd938-19d8-49a2-a50e-1f18a7e4f194 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:33.443933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbbd938-19d8-49a2-a50e-1f18a7e4f194 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:33.446026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbbd938-19d8-49a2-a50e-1f18a7e4f194 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:33.469184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1207/2397] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:25:33 2022] POST /identity/v3/users => generated 281 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:25:33.473114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d629e1-eada-40f1-a83d-ebf03d072b88 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:33.484325 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d629e1-eada-40f1-a83d-ebf03d072b88 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:33.489028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6dd430-b800-4068-98e3-bc24a3725ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:33.489750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6dd430-b800-4068-98e3-bc24a3725ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:33.489997 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6dd430-b800-4068-98e3-bc24a3725ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:33.490305 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6dd430-b800-4068-98e3-bc24a3725ce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:33.490955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6dd430-b800-4068-98e3-bc24a3725ce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:33.501308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6dd430-b800-4068-98e3-bc24a3725ce7 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:33.503568 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1191/2398] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:33.507960 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbbd938-19d8-49a2-a50e-1f18a7e4f194 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:33.518549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbbd938-19d8-49a2-a50e-1f18a7e4f194 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:33.523727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c107b71-ba69-4679-a16f-d9b8c425b468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:33.524338 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c107b71-ba69-4679-a16f-d9b8c425b468 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:33.524655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c107b71-ba69-4679-a16f-d9b8c425b468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:33.524880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c107b71-ba69-4679-a16f-d9b8c425b468 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:33.525556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c107b71-ba69-4679-a16f-d9b8c425b468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:33.527131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c107b71-ba69-4679-a16f-d9b8c425b468 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:33.546313 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1208/2399] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:25:33.546988 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6dd430-b800-4068-98e3-bc24a3725ce7 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:33.560473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6dd430-b800-4068-98e3-bc24a3725ce7 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:33.566647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44298426-b4f9-42be-9cda-cb85f7fc8192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:33.568382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44298426-b4f9-42be-9cda-cb85f7fc8192 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:33.568709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44298426-b4f9-42be-9cda-cb85f7fc8192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:33.569292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44298426-b4f9-42be-9cda-cb85f7fc8192 admin admin] PATH_INFO: `/v3/projects/c43e6c5fa08e4ff39c2b29c1fbff266d/users/25b6cc8e0a014cec86cfe2635cd2a75e/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:33.588823 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44298426-b4f9-42be-9cda-cb85f7fc8192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c43e6c5fa08e4ff39c2b29c1fbff266d, user_id=25b6cc8e0a014cec86cfe2635cd2a75e, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:33.591628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44298426-b4f9-42be-9cda-cb85f7fc8192 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:33.635624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b540e24-a2ef-4262-ae85-39bae14a6896 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:33.635844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1192/2400] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:25:33 2022] PUT /identity/v3/projects/c43e6c5fa08e4ff39c2b29c1fbff266d/users/25b6cc8e0a014cec86cfe2635cd2a75e/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:25:33.636106 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b540e24-a2ef-4262-ae85-39bae14a6896 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:33.636453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b540e24-a2ef-4262-ae85-39bae14a6896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:33.672596 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-3b540e24-a2ef-4262-ae85-39bae14a6896 None None] MFA Rules not processed for user `25b6cc8e0a014cec86cfe2635cd2a75e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:33.674428 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-3b540e24-a2ef-4262-ae85-39bae14a6896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:33.689606 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:33.768636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44298426-b4f9-42be-9cda-cb85f7fc8192 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:33.770504 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-44298426-b4f9-42be-9cda-cb85f7fc8192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:33.780354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1209/2401] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:25:33.888588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b540e24-a2ef-4262-ae85-39bae14a6896 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:33.890706 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-3b540e24-a2ef-4262-ae85-39bae14a6896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:33.899445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44298426-b4f9-42be-9cda-cb85f7fc8192 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:33.905506 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3e19a-6ea1-48f5-9ade-a17dca367dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:33.906245 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3e19a-6ea1-48f5-9ade-a17dca367dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:33.906591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3e19a-6ea1-48f5-9ade-a17dca367dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:33.906933 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3e19a-6ea1-48f5-9ade-a17dca367dcb admin admin] PATH_INFO: `/v3/users/ec3dc4e21d564618afb548369cd214c1` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:33.908431 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3e19a-6ea1-48f5-9ade-a17dca367dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec3dc4e21d564618afb548369cd214c1)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:33.910978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3e19a-6ea1-48f5-9ade-a17dca367dcb admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:33.972136 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b540e24-a2ef-4262-ae85-39bae14a6896 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:33.973018 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d4c3e19a-6ea1-48f5-9ade-a17dca367dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec3dc4e21d564618afb548369cd214c1'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:33.973709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d4c3e19a-6ea1-48f5-9ade-a17dca367dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec3dc4e21d564618afb548369cd214c1'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:33.976155 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39c3ba-c6cc-4ce9-bbc5-7633fb673dbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:33.976864 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39c3ba-c6cc-4ce9-bbc5-7633fb673dbf service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:33.977246 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39c3ba-c6cc-4ce9-bbc5-7633fb673dbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:33.977590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39c3ba-c6cc-4ce9-bbc5-7633fb673dbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:33.979544 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2a39c3ba-c6cc-4ce9-bbc5-7633fb673dbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:33.988066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d4c3e19a-6ea1-48f5-9ade-a17dca367dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec3dc4e21d564618afb548369cd214c1'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:33.993893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39c3ba-c6cc-4ce9-bbc5-7633fb673dbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:33.995607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39c3ba-c6cc-4ce9-bbc5-7633fb673dbf service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:34.002792 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d4c3e19a-6ea1-48f5-9ade-a17dca367dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec3dc4e21d564618afb548369cd214c1'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:34.003591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d4c3e19a-6ea1-48f5-9ade-a17dca367dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec3dc4e21d564618afb548369cd214c1'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:34.008377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1210/2402] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:25:34.016277 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:INSERT=4,keystone:DELETE=13,keystone:SELECT=83 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:34.024817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1193/2403] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:25:33 2022] DELETE /identity/v3/users/ec3dc4e21d564618afb548369cd214c1 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:25:34.027295 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39c3ba-c6cc-4ce9-bbc5-7633fb673dbf service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:34.028973 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2a39c3ba-c6cc-4ce9-bbc5-7633fb673dbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:34.029322 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5e3a1-7c52-4913-a47a-68774339d46a None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:34.029603 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5e3a1-7c52-4913-a47a-68774339d46a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:34.029881 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5e3a1-7c52-4913-a47a-68774339d46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:34.046605 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39c3ba-c6cc-4ce9-bbc5-7633fb673dbf service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:34.080427 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-9ee5e3a1-7c52-4913-a47a-68774339d46a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:34.090417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-9ee5e3a1-7c52-4913-a47a-68774339d46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:34.154701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23505d37-df3a-4e65-a228-9cbdaae06013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:34.155430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23505d37-df3a-4e65-a228-9cbdaae06013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:34.155620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23505d37-df3a-4e65-a228-9cbdaae06013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:34.155880 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23505d37-df3a-4e65-a228-9cbdaae06013 admin admin] PATH_INFO: `/v3/users/25b6cc8e0a014cec86cfe2635cd2a75e` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:34.157210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23505d37-df3a-4e65-a228-9cbdaae06013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25b6cc8e0a014cec86cfe2635cd2a75e)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:34.159291 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23505d37-df3a-4e65-a228-9cbdaae06013 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:34.204872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1194/2404] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:25:34.217872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5e3a1-7c52-4913-a47a-68774339d46a None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:34.220442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-9ee5e3a1-7c52-4913-a47a-68774339d46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:34.287057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-23505d37-df3a-4e65-a228-9cbdaae06013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25b6cc8e0a014cec86cfe2635cd2a75e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:34.288282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-23505d37-df3a-4e65-a228-9cbdaae06013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25b6cc8e0a014cec86cfe2635cd2a75e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:34.303859 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-23505d37-df3a-4e65-a228-9cbdaae06013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25b6cc8e0a014cec86cfe2635cd2a75e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:34.304599 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-23505d37-df3a-4e65-a228-9cbdaae06013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25b6cc8e0a014cec86cfe2635cd2a75e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:34.311370 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5e3a1-7c52-4913-a47a-68774339d46a None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:34.315061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:34.323689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-23505d37-df3a-4e65-a228-9cbdaae06013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25b6cc8e0a014cec86cfe2635cd2a75e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:34.324123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309f365-b739-4feb-8694-489f1c3ff2c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:34.324783 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309f365-b739-4feb-8694-489f1c3ff2c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:34.325070 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309f365-b739-4feb-8694-489f1c3ff2c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:34.326348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309f365-b739-4feb-8694-489f1c3ff2c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:34.330722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b309f365-b739-4feb-8694-489f1c3ff2c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:34.359464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1211/2405] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:25:34 2022] DELETE /identity/v3/users/25b6cc8e0a014cec86cfe2635cd2a75e => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:34.362508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbf602-f8a5-4084-958e-73d8e67af75f None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:34.362947 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbf602-f8a5-4084-958e-73d8e67af75f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:34.363493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbf602-f8a5-4084-958e-73d8e67af75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:34.364126 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309f365-b739-4feb-8694-489f1c3ff2c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:34.370337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309f365-b739-4feb-8694-489f1c3ff2c1 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:34.382269 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1195/2406] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:25:34.401652 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-9fbbf602-f8a5-4084-958e-73d8e67af75f None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:25:34.403508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9fbbf602-f8a5-4084-958e-73d8e67af75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:34.533205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1212/2407] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:25:34.555190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309f365-b739-4feb-8694-489f1c3ff2c1 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:34.557361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b309f365-b739-4feb-8694-489f1c3ff2c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:34.573156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309f365-b739-4feb-8694-489f1c3ff2c1 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:34.687830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0ee3e-1bb8-4c9c-82e5-5fd4c62363b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:34.688498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0ee3e-1bb8-4c9c-82e5-5fd4c62363b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:34.690752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0ee3e-1bb8-4c9c-82e5-5fd4c62363b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:34.691057 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0ee3e-1bb8-4c9c-82e5-5fd4c62363b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:34.713323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe0ee3e-1bb8-4c9c-82e5-5fd4c62363b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:34.714802 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe0ee3e-1bb8-4c9c-82e5-5fd4c62363b9 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:34.739033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1196/2408] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:25:35.150740 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbf602-f8a5-4084-958e-73d8e67af75f None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:35.152707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9fbbf602-f8a5-4084-958e-73d8e67af75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:35.176617 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbf602-f8a5-4084-958e-73d8e67af75f None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:35.189809 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019ffb1-05ba-4a54-b0c4-d2128e32637e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:35.190600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019ffb1-05ba-4a54-b0c4-d2128e32637e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:35.190998 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019ffb1-05ba-4a54-b0c4-d2128e32637e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:35.191417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019ffb1-05ba-4a54-b0c4-d2128e32637e admin admin] PATH_INFO: `/v3/projects/937511abf270462ba94a4b56e0594b51` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:35.192736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019ffb1-05ba-4a54-b0c4-d2128e32637e admin admin] RBAC: Authorizing `identity:delete_project(project_id=937511abf270462ba94a4b56e0594b51)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:35.194894 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019ffb1-05ba-4a54-b0c4-d2128e32637e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:35.274297 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d019ffb1-05ba-4a54-b0c4-d2128e32637e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '937511abf270462ba94a4b56e0594b51', 'request_id': 'req-d019ffb1-05ba-4a54-b0c4-d2128e32637e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:35.289267 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d019ffb1-05ba-4a54-b0c4-d2128e32637e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '937511abf270462ba94a4b56e0594b51', 'request_id': 'req-d019ffb1-05ba-4a54-b0c4-d2128e32637e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:35.341845 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d019ffb1-05ba-4a54-b0c4-d2128e32637e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '937511abf270462ba94a4b56e0594b51', 'request_id': 'req-d019ffb1-05ba-4a54-b0c4-d2128e32637e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:35.382449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d019ffb1-05ba-4a54-b0c4-d2128e32637e admin admin] The token cache is being invalidate because project 937511abf270462ba94a4b56e0594b51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:25:35.382449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d019ffb1-05ba-4a54-b0c4-d2128e32637e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:35.382449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d019ffb1-05ba-4a54-b0c4-d2128e32637e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:35.384501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1213/2409] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:25:35 2022] DELETE /identity/v3/projects/937511abf270462ba94a4b56e0594b51 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:35.388469 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0ee3e-1bb8-4c9c-82e5-5fd4c62363b9 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:35.402301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0ee3e-1bb8-4c9c-82e5-5fd4c62363b9 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:35.421556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019ffb1-05ba-4a54-b0c4-d2128e32637e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:35.423142 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d019ffb1-05ba-4a54-b0c4-d2128e32637e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:35.472818 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:35.494651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938d829-3955-4cf1-8ce1-48b16de1f53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:35.495296 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938d829-3955-4cf1-8ce1-48b16de1f53b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:35.495579 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938d829-3955-4cf1-8ce1-48b16de1f53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:35.495997 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938d829-3955-4cf1-8ce1-48b16de1f53b admin admin] PATH_INFO: `/v3/users/50926e9eb0de4cb5bdd887cbb3ad34cd` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:35.497348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938d829-3955-4cf1-8ce1-48b16de1f53b admin admin] RBAC: Authorizing `identity:delete_user(user_id=50926e9eb0de4cb5bdd887cbb3ad34cd)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:35.499422 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938d829-3955-4cf1-8ce1-48b16de1f53b admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:35.503181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019ffb1-05ba-4a54-b0c4-d2128e32637e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:35.509029 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c96f9-41d4-4c2a-ac58-d66807d5becc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:35.509806 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c96f9-41d4-4c2a-ac58-d66807d5becc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:35.510153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c96f9-41d4-4c2a-ac58-d66807d5becc admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:35.510500 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c96f9-41d4-4c2a-ac58-d66807d5becc admin admin] PATH_INFO: `/v3/projects/c43e6c5fa08e4ff39c2b29c1fbff266d` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:35.511710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89c96f9-41d4-4c2a-ac58-d66807d5becc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c43e6c5fa08e4ff39c2b29c1fbff266d)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:35.513770 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89c96f9-41d4-4c2a-ac58-d66807d5becc admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:35.515766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:35.584582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e89c96f9-41d4-4c2a-ac58-d66807d5becc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c43e6c5fa08e4ff39c2b29c1fbff266d', 'request_id': 'req-e89c96f9-41d4-4c2a-ac58-d66807d5becc'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:35.592449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e89c96f9-41d4-4c2a-ac58-d66807d5becc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c43e6c5fa08e4ff39c2b29c1fbff266d', 'request_id': 'req-e89c96f9-41d4-4c2a-ac58-d66807d5becc'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:35.594540 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-a938d829-3955-4cf1-8ce1-48b16de1f53b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50926e9eb0de4cb5bdd887cbb3ad34cd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:35.595381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-a938d829-3955-4cf1-8ce1-48b16de1f53b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50926e9eb0de4cb5bdd887cbb3ad34cd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:35.602898 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e89c96f9-41d4-4c2a-ac58-d66807d5becc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c43e6c5fa08e4ff39c2b29c1fbff266d', 'request_id': 'req-e89c96f9-41d4-4c2a-ac58-d66807d5becc'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:35.615699 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-a938d829-3955-4cf1-8ce1-48b16de1f53b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50926e9eb0de4cb5bdd887cbb3ad34cd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:35.623160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e89c96f9-41d4-4c2a-ac58-d66807d5becc admin admin] The token cache is being invalidate because project c43e6c5fa08e4ff39c2b29c1fbff266d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:25:35.624115 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e89c96f9-41d4-4c2a-ac58-d66807d5becc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:35.624410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e89c96f9-41d4-4c2a-ac58-d66807d5becc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:35.625594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-a938d829-3955-4cf1-8ce1-48b16de1f53b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50926e9eb0de4cb5bdd887cbb3ad34cd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:35.626670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-a938d829-3955-4cf1-8ce1-48b16de1f53b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50926e9eb0de4cb5bdd887cbb3ad34cd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:35.628804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1214/2410] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:25:35 2022] DELETE /identity/v3/projects/c43e6c5fa08e4ff39c2b29c1fbff266d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:25:35.674832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1197/2411] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:25:35 2022] DELETE /identity/v3/users/50926e9eb0de4cb5bdd887cbb3ad34cd => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:25:36.086470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c96f9-41d4-4c2a-ac58-d66807d5becc admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:36.088023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e89c96f9-41d4-4c2a-ac58-d66807d5becc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:36.155346 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c96f9-41d4-4c2a-ac58-d66807d5becc admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:36.212058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708e60e-b575-42a7-bfc6-abaeeb43c12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:36.212728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708e60e-b575-42a7-bfc6-abaeeb43c12e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:36.212959 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708e60e-b575-42a7-bfc6-abaeeb43c12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:36.213274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d708e60e-b575-42a7-bfc6-abaeeb43c12e admin admin] PATH_INFO: `/v3/projects/72710bb57e4842fbbfa46abad4353e85` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:36.214389 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708e60e-b575-42a7-bfc6-abaeeb43c12e admin admin] RBAC: Authorizing `identity:delete_project(project_id=72710bb57e4842fbbfa46abad4353e85)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:36.216271 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d708e60e-b575-42a7-bfc6-abaeeb43c12e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:36.255486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:36.264040 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d708e60e-b575-42a7-bfc6-abaeeb43c12e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72710bb57e4842fbbfa46abad4353e85', 'request_id': 'req-d708e60e-b575-42a7-bfc6-abaeeb43c12e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:36.270145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d708e60e-b575-42a7-bfc6-abaeeb43c12e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72710bb57e4842fbbfa46abad4353e85', 'request_id': 'req-d708e60e-b575-42a7-bfc6-abaeeb43c12e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:36.277399 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d708e60e-b575-42a7-bfc6-abaeeb43c12e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72710bb57e4842fbbfa46abad4353e85', 'request_id': 'req-d708e60e-b575-42a7-bfc6-abaeeb43c12e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:36.287721 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d708e60e-b575-42a7-bfc6-abaeeb43c12e admin admin] The token cache is being invalidate because project 72710bb57e4842fbbfa46abad4353e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:25:36.288010 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d708e60e-b575-42a7-bfc6-abaeeb43c12e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:36.288643 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d708e60e-b575-42a7-bfc6-abaeeb43c12e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:36.290168 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1215/2412] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:25:36 2022] DELETE /identity/v3/projects/72710bb57e4842fbbfa46abad4353e85 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:36.294187 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938d829-3955-4cf1-8ce1-48b16de1f53b admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:36.295593 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-a938d829-3955-4cf1-8ce1-48b16de1f53b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:36.308815 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938d829-3955-4cf1-8ce1-48b16de1f53b admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:36.365842 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b44252-2273-4f16-8a98-ef6183aad3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:36.366488 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b44252-2273-4f16-8a98-ef6183aad3c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:36.366782 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b44252-2273-4f16-8a98-ef6183aad3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:36.367050 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b44252-2273-4f16-8a98-ef6183aad3c2 admin admin] PATH_INFO: `/v3/users/e11003b9c9414e17bc9cc03f949abe20` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:36.368212 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b44252-2273-4f16-8a98-ef6183aad3c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e11003b9c9414e17bc9cc03f949abe20)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:36.370281 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b44252-2273-4f16-8a98-ef6183aad3c2 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:36.415978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-32b44252-2273-4f16-8a98-ef6183aad3c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e11003b9c9414e17bc9cc03f949abe20'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:36.416683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-32b44252-2273-4f16-8a98-ef6183aad3c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e11003b9c9414e17bc9cc03f949abe20'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:36.429076 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-32b44252-2273-4f16-8a98-ef6183aad3c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e11003b9c9414e17bc9cc03f949abe20'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:36.430693 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:36.436586 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-32b44252-2273-4f16-8a98-ef6183aad3c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e11003b9c9414e17bc9cc03f949abe20'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:36.437278 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-32b44252-2273-4f16-8a98-ef6183aad3c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e11003b9c9414e17bc9cc03f949abe20'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:36.450265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1198/2413] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:25:36 2022] DELETE /identity/v3/users/e11003b9c9414e17bc9cc03f949abe20 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:36.715746 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708e60e-b575-42a7-bfc6-abaeeb43c12e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:36.717275 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d708e60e-b575-42a7-bfc6-abaeeb43c12e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:36.749198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d708e60e-b575-42a7-bfc6-abaeeb43c12e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:36.812319 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d91ce-2263-4bc8-947f-329c9acf277e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:36.812919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d91ce-2263-4bc8-947f-329c9acf277e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:36.813231 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d91ce-2263-4bc8-947f-329c9acf277e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:36.813514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d91ce-2263-4bc8-947f-329c9acf277e admin admin] PATH_INFO: `/v3/projects/724c780ea76f4d0cbdb787a17e565468` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:36.814756 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d91ce-2263-4bc8-947f-329c9acf277e admin admin] RBAC: Authorizing `identity:delete_project(project_id=724c780ea76f4d0cbdb787a17e565468)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:36.818071 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d91ce-2263-4bc8-947f-329c9acf277e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:36.866742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-982d91ce-2263-4bc8-947f-329c9acf277e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '724c780ea76f4d0cbdb787a17e565468', 'request_id': 'req-982d91ce-2263-4bc8-947f-329c9acf277e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:36.873065 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-982d91ce-2263-4bc8-947f-329c9acf277e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '724c780ea76f4d0cbdb787a17e565468', 'request_id': 'req-982d91ce-2263-4bc8-947f-329c9acf277e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:36.881486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-982d91ce-2263-4bc8-947f-329c9acf277e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '724c780ea76f4d0cbdb787a17e565468', 'request_id': 'req-982d91ce-2263-4bc8-947f-329c9acf277e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:36.893797 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-982d91ce-2263-4bc8-947f-329c9acf277e admin admin] The token cache is being invalidate because project 724c780ea76f4d0cbdb787a17e565468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:25:36.894093 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-982d91ce-2263-4bc8-947f-329c9acf277e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:36.894749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-982d91ce-2263-4bc8-947f-329c9acf277e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:25:36.897073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1216/2414] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:25:36 2022] DELETE /identity/v3/projects/724c780ea76f4d0cbdb787a17e565468 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:25:43.244183 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b44252-2273-4f16-8a98-ef6183aad3c2 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:43.245620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-32b44252-2273-4f16-8a98-ef6183aad3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:43.303294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b44252-2273-4f16-8a98-ef6183aad3c2 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:43.350793 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b95ae-73ef-4cc8-b9f7-6f08e405487e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:43.351417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b95ae-73ef-4cc8-b9f7-6f08e405487e service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:43.351698 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b95ae-73ef-4cc8-b9f7-6f08e405487e service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:43.351970 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b95ae-73ef-4cc8-b9f7-6f08e405487e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:43.353735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-547b95ae-73ef-4cc8-b9f7-6f08e405487e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:43.415520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b95ae-73ef-4cc8-b9f7-6f08e405487e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:43.416948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b95ae-73ef-4cc8-b9f7-6f08e405487e service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:43.426540 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1199/2415] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:25:46.890174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=61,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:53.422158 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=76 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:25:56.009005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d91ce-2263-4bc8-947f-329c9acf277e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:25:56.011410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-982d91ce-2263-4bc8-947f-329c9acf277e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:56.085692 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d91ce-2263-4bc8-947f-329c9acf277e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:25:56.093759 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d445cc3-e152-49e2-b7b3-b34270838efd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:25:56.094380 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d445cc3-e152-49e2-b7b3-b34270838efd service placement] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:25:56.094661 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d445cc3-e152-49e2-b7b3-b34270838efd service placement] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:25:56.095071 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d445cc3-e152-49e2-b7b3-b34270838efd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:25:56.097119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9d445cc3-e152-49e2-b7b3-b34270838efd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:25:56.109473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d445cc3-e152-49e2-b7b3-b34270838efd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:25:56.111040 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d445cc3-e152-49e2-b7b3-b34270838efd service placement] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:25:56.121753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1217/2416] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:25:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:26:05.583207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feb8ae-2128-4800-99b3-84a67c800a92 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:05.583733 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feb8ae-2128-4800-99b3-84a67c800a92 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:05.583733 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2feb8ae-2128-4800-99b3-84a67c800a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:05.603219 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-c2feb8ae-2128-4800-99b3-84a67c800a92 None None] MFA Rules not processed for user `bc69cf267bea41bf890bc7551d84ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:26:05.604407 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c2feb8ae-2128-4800-99b3-84a67c800a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:05.659360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1200/2417] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 12:26:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:26:05.681558 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d445cc3-e152-49e2-b7b3-b34270838efd service placement] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:05.683145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9d445cc3-e152-49e2-b7b3-b34270838efd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:05.753648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d445cc3-e152-49e2-b7b3-b34270838efd service placement] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:05.758978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211b4e9-1dd6-491b-aba6-a9d5ee0ef481 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975a8d7349994520bcbfb5f75e52aac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:05.759580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211b4e9-1dd6-491b-aba6-a9d5ee0ef481 service swift] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:05.759856 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211b4e9-1dd6-491b-aba6-a9d5ee0ef481 service swift] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:05.760122 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211b4e9-1dd6-491b-aba6-a9d5ee0ef481 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:05.767995 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211b4e9-1dd6-491b-aba6-a9d5ee0ef481 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:05.769627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211b4e9-1dd6-491b-aba6-a9d5ee0ef481 service swift] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:05.777528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1218/2418] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 12:26:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 12:26:06.296413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb6b17-31ac-4067-b439-2ca25c723a7c None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:06.296704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb6b17-31ac-4067-b439-2ca25c723a7c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:06.296978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb6b17-31ac-4067-b439-2ca25c723a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:06.315764 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-cdbb6b17-31ac-4067-b439-2ca25c723a7c None None] MFA Rules not processed for user `bc69cf267bea41bf890bc7551d84ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:26:06.317065 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-cdbb6b17-31ac-4067-b439-2ca25c723a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:06.324455 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1201/2419] 10.222.0.60 () {58 vars in 967 bytes} [Wed Sep 28 12:26:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:26:06.337119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211b4e9-1dd6-491b-aba6-a9d5ee0ef481 service swift] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:06.348942 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211b4e9-1dd6-491b-aba6-a9d5ee0ef481 service swift] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:06.353957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c904db-b4dd-498a-8e9a-b030eca8848f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975a8d7349994520bcbfb5f75e52aac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:06.354551 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c904db-b4dd-498a-8e9a-b030eca8848f service swift] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:06.354829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c904db-b4dd-498a-8e9a-b030eca8848f service swift] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:06.355099 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c904db-b4dd-498a-8e9a-b030eca8848f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:06.363032 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c904db-b4dd-498a-8e9a-b030eca8848f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:06.364832 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c904db-b4dd-498a-8e9a-b030eca8848f service swift] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:06.372816 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1219/2420] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 12:26:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 12:26:06.605560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe093763-b61b-4dfa-8daf-464b4bb1a8b1 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:06.606058 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe093763-b61b-4dfa-8daf-464b4bb1a8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:06.606284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe093763-b61b-4dfa-8daf-464b4bb1a8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:06.626340 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-fe093763-b61b-4dfa-8daf-464b4bb1a8b1 None None] MFA Rules not processed for user `bc69cf267bea41bf890bc7551d84ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:26:06.627527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-fe093763-b61b-4dfa-8daf-464b4bb1a8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:06.638214 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1202/2421] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 12:26:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:26:06.650628 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c904db-b4dd-498a-8e9a-b030eca8848f service swift] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:06.660793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c904db-b4dd-498a-8e9a-b030eca8848f service swift] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:06.665639 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003a5c6-9879-4f51-8f75-6db8b31894c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975a8d7349994520bcbfb5f75e52aac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:06.666226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003a5c6-9879-4f51-8f75-6db8b31894c0 service swift] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:06.666502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003a5c6-9879-4f51-8f75-6db8b31894c0 service swift] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:06.666768 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003a5c6-9879-4f51-8f75-6db8b31894c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:06.674495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003a5c6-9879-4f51-8f75-6db8b31894c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:06.676068 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b003a5c6-9879-4f51-8f75-6db8b31894c0 service swift] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:06.683788 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1220/2422] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 12:26:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 12:26:07.002709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fdf130-1dff-4a31-ae8a-876db3d3fb13 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:07.002992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fdf130-1dff-4a31-ae8a-876db3d3fb13 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:07.003271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fdf130-1dff-4a31-ae8a-876db3d3fb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:07.024378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-c7fdf130-1dff-4a31-ae8a-876db3d3fb13 None None] MFA Rules not processed for user `bc69cf267bea41bf890bc7551d84ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:26:07.025606 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c7fdf130-1dff-4a31-ae8a-876db3d3fb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:07.032662 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1203/2423] 10.222.0.60 () {58 vars in 967 bytes} [Wed Sep 28 12:26:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:26:07.045300 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003a5c6-9879-4f51-8f75-6db8b31894c0 service swift] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:07.055949 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b003a5c6-9879-4f51-8f75-6db8b31894c0 service swift] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:07.061008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd41001-c1f4-4676-8e46-661d3d6243a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975a8d7349994520bcbfb5f75e52aac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:07.061634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd41001-c1f4-4676-8e46-661d3d6243a4 service swift] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:07.061919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd41001-c1f4-4676-8e46-661d3d6243a4 service swift] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:07.062194 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd41001-c1f4-4676-8e46-661d3d6243a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:07.070474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd41001-c1f4-4676-8e46-661d3d6243a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:07.072022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd41001-c1f4-4676-8e46-661d3d6243a4 service swift] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:07.079836 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1221/2424] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 12:26:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 12:26:09.185959 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fdf130-1dff-4a31-ae8a-876db3d3fb13 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:09.187490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c7fdf130-1dff-4a31-ae8a-876db3d3fb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:09.213422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd41001-c1f4-4676-8e46-661d3d6243a4 service swift] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:09.215033 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-ccd41001-c1f4-4676-8e46-661d3d6243a4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:09.228492 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd41001-c1f4-4676-8e46-661d3d6243a4 service swift] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:09.232389 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d007c-a59f-4f8c-893f-3764d18584be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:09.232917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d007c-a59f-4f8c-893f-3764d18584be service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:09.233220 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d007c-a59f-4f8c-893f-3764d18584be service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:09.233494 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d007c-a59f-4f8c-893f-3764d18584be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:09.235445 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-814d007c-a59f-4f8c-893f-3764d18584be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:09.256656 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fdf130-1dff-4a31-ae8a-876db3d3fb13 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:09.261916 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570905da-e618-44d8-9bcb-209614d4088d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:09.262576 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570905da-e618-44d8-9bcb-209614d4088d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:09.262856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570905da-e618-44d8-9bcb-209614d4088d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:09.263130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570905da-e618-44d8-9bcb-209614d4088d admin admin] PATH_INFO: `/v3/users/d12d46f42d73486fbbc3a1c2df0f3ab9` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:09.264268 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570905da-e618-44d8-9bcb-209614d4088d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d12d46f42d73486fbbc3a1c2df0f3ab9)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:09.266210 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570905da-e618-44d8-9bcb-209614d4088d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:09.279093 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:26:09.307637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d007c-a59f-4f8c-893f-3764d18584be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:09.309232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d007c-a59f-4f8c-893f-3764d18584be service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:09.318385 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-570905da-e618-44d8-9bcb-209614d4088d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd12d46f42d73486fbbc3a1c2df0f3ab9'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:09.319049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-570905da-e618-44d8-9bcb-209614d4088d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd12d46f42d73486fbbc3a1c2df0f3ab9'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:09.320074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1222/2425] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:26:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:26:09.331338 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-570905da-e618-44d8-9bcb-209614d4088d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd12d46f42d73486fbbc3a1c2df0f3ab9'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:09.339172 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-570905da-e618-44d8-9bcb-209614d4088d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd12d46f42d73486fbbc3a1c2df0f3ab9'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:09.339840 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-570905da-e618-44d8-9bcb-209614d4088d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd12d46f42d73486fbbc3a1c2df0f3ab9'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:09.352955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1204/2426] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:26:09 2022] DELETE /identity/v3/users/d12d46f42d73486fbbc3a1c2df0f3ab9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:26:09.357569 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b659871-c2dd-4a0e-b491-90f038009fa7 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:09.357855 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b659871-c2dd-4a0e-b491-90f038009fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:09.358130 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b659871-c2dd-4a0e-b491-90f038009fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:09.378562 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-1b659871-c2dd-4a0e-b491-90f038009fa7 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:26:09.380454 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-1b659871-c2dd-4a0e-b491-90f038009fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:09.443709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1223/2427] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:26:09.453504 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570905da-e618-44d8-9bcb-209614d4088d admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:09.454982 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-570905da-e618-44d8-9bcb-209614d4088d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:09.462598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:26:09.520635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570905da-e618-44d8-9bcb-209614d4088d admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:09.524361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88593ab4-bf2a-4194-b7ff-51aeff81c679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:09.525031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88593ab4-bf2a-4194-b7ff-51aeff81c679 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:09.525324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88593ab4-bf2a-4194-b7ff-51aeff81c679 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:09.525585 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88593ab4-bf2a-4194-b7ff-51aeff81c679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:09.535222 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88593ab4-bf2a-4194-b7ff-51aeff81c679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:09.536938 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88593ab4-bf2a-4194-b7ff-51aeff81c679 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:09.548460 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1205/2428] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:26:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:26:09.763052 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b659871-c2dd-4a0e-b491-90f038009fa7 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:09.764720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-1b659871-c2dd-4a0e-b491-90f038009fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:09.783096 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b659871-c2dd-4a0e-b491-90f038009fa7 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:09.792468 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4489d2-8adb-4751-98c6-ce54a549377e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:09.793016 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4489d2-8adb-4751-98c6-ce54a549377e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:09.793376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4489d2-8adb-4751-98c6-ce54a549377e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:09.793658 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4489d2-8adb-4751-98c6-ce54a549377e admin admin] PATH_INFO: `/v3/projects/14944124ab8a4487b879592a02981d12` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:09.795010 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4489d2-8adb-4751-98c6-ce54a549377e admin admin] RBAC: Authorizing `identity:delete_project(project_id=14944124ab8a4487b879592a02981d12)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:09.800108 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4489d2-8adb-4751-98c6-ce54a549377e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:09.868293 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-ba4489d2-8adb-4751-98c6-ce54a549377e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14944124ab8a4487b879592a02981d12', 'request_id': 'req-ba4489d2-8adb-4751-98c6-ce54a549377e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:09.875359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-ba4489d2-8adb-4751-98c6-ce54a549377e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14944124ab8a4487b879592a02981d12', 'request_id': 'req-ba4489d2-8adb-4751-98c6-ce54a549377e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:09.883923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-ba4489d2-8adb-4751-98c6-ce54a549377e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14944124ab8a4487b879592a02981d12', 'request_id': 'req-ba4489d2-8adb-4751-98c6-ce54a549377e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:09.896807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-ba4489d2-8adb-4751-98c6-ce54a549377e admin admin] The token cache is being invalidate because project 14944124ab8a4487b879592a02981d12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:26:09.897816 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-ba4489d2-8adb-4751-98c6-ce54a549377e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:09.898556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-ba4489d2-8adb-4751-98c6-ce54a549377e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:09.900249 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1224/2429] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:26:09 2022] DELETE /identity/v3/projects/14944124ab8a4487b879592a02981d12 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:26:09.905262 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9234bd9-b541-4cbe-93a4-da1c4c4c7f6d None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:09.905579 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9234bd9-b541-4cbe-93a4-da1c4c4c7f6d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:09.905889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9234bd9-b541-4cbe-93a4-da1c4c4c7f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:09.926318 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-a9234bd9-b541-4cbe-93a4-da1c4c4c7f6d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:26:09.927648 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-a9234bd9-b541-4cbe-93a4-da1c4c4c7f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:09.987457 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1206/2430] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:26:09.995584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01912a84-e51d-4cb4-8d83-2bba46bbf21e None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:09.995877 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01912a84-e51d-4cb4-8d83-2bba46bbf21e None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:09.996159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01912a84-e51d-4cb4-8d83-2bba46bbf21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:10.016392 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-01912a84-e51d-4cb4-8d83-2bba46bbf21e None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:26:10.017634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-01912a84-e51d-4cb4-8d83-2bba46bbf21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:10.027576 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1225/2431] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:26:10.031646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9234bd9-b541-4cbe-93a4-da1c4c4c7f6d None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:10.044013 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9234bd9-b541-4cbe-93a4-da1c4c4c7f6d None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:10.049109 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5178e33-660b-4b4c-bf76-3f463254785e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:10.049771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5178e33-660b-4b4c-bf76-3f463254785e admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:10.050048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5178e33-660b-4b4c-bf76-3f463254785e admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:10.050824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5178e33-660b-4b4c-bf76-3f463254785e admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:10.051508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5178e33-660b-4b4c-bf76-3f463254785e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:26:10.051876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5178e33-660b-4b4c-bf76-3f463254785e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:10.053593 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5178e33-660b-4b4c-bf76-3f463254785e admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:10.070055 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1207/2432] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:26:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:26:10.071407 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01912a84-e51d-4cb4-8d83-2bba46bbf21e None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:10.081080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01912a84-e51d-4cb4-8d83-2bba46bbf21e None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:10.086381 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da94964-3559-44a3-b373-0f514a20c13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:10.087008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da94964-3559-44a3-b373-0f514a20c13b admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:10.087288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da94964-3559-44a3-b373-0f514a20c13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:10.087584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da94964-3559-44a3-b373-0f514a20c13b admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:10.088194 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da94964-3559-44a3-b373-0f514a20c13b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:10.090157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da94964-3559-44a3-b373-0f514a20c13b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:10.123167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1226/2433] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:26:10 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:26:10.124364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5178e33-660b-4b4c-bf76-3f463254785e admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:10.140987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5178e33-660b-4b4c-bf76-3f463254785e admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:10.212381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da577b4d-cfbf-4682-a4a5-63e608ddf689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:10.213077 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da577b4d-cfbf-4682-a4a5-63e608ddf689 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:10.213338 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da577b4d-cfbf-4682-a4a5-63e608ddf689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:10.213609 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da577b4d-cfbf-4682-a4a5-63e608ddf689 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:10.214240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da577b4d-cfbf-4682-a4a5-63e608ddf689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:10.216159 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da577b4d-cfbf-4682-a4a5-63e608ddf689 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:10.244474 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1208/2434] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:26:10 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:26:10.245697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da94964-3559-44a3-b373-0f514a20c13b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:10.256521 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da94964-3559-44a3-b373-0f514a20c13b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:10.261159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88813d51-8219-41dd-abe4-1c13a4bc0983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:10.261772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88813d51-8219-41dd-abe4-1c13a4bc0983 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:10.262048 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88813d51-8219-41dd-abe4-1c13a4bc0983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:10.262318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88813d51-8219-41dd-abe4-1c13a4bc0983 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:10.262953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88813d51-8219-41dd-abe4-1c13a4bc0983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:10.264538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88813d51-8219-41dd-abe4-1c13a4bc0983 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:10.275155 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1227/2435] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:26:10.279510 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da577b4d-cfbf-4682-a4a5-63e608ddf689 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:10.290926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da577b4d-cfbf-4682-a4a5-63e608ddf689 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:10.296194 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d1273-65ba-4c81-bcf5-381530bc7a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:10.296812 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d1273-65ba-4c81-bcf5-381530bc7a14 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:10.297122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d1273-65ba-4c81-bcf5-381530bc7a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:10.297468 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d1273-65ba-4c81-bcf5-381530bc7a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:10.298105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d1273-65ba-4c81-bcf5-381530bc7a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:10.299739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d1273-65ba-4c81-bcf5-381530bc7a14 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:10.312419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1209/2436] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:26:10.316928 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88813d51-8219-41dd-abe4-1c13a4bc0983 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:10.327032 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88813d51-8219-41dd-abe4-1c13a4bc0983 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:10.331563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37299620-54c6-45d2-b8b4-89714b8cae7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:10.332215 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37299620-54c6-45d2-b8b4-89714b8cae7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:10.332516 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37299620-54c6-45d2-b8b4-89714b8cae7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:10.332786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37299620-54c6-45d2-b8b4-89714b8cae7d admin admin] PATH_INFO: `/v3/projects/4a803e0aefca45b3b0756c3d451c4473/users/1728596610fe48f49f459aab2c1eef61/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:10.347911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37299620-54c6-45d2-b8b4-89714b8cae7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a803e0aefca45b3b0756c3d451c4473, user_id=1728596610fe48f49f459aab2c1eef61, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:10.350558 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37299620-54c6-45d2-b8b4-89714b8cae7d admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:10.352807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:26:10.367636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1228/2437] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:26:10 2022] PUT /identity/v3/projects/4a803e0aefca45b3b0756c3d451c4473/users/1728596610fe48f49f459aab2c1eef61/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:26:10.378604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04190331-3e12-4135-b459-4797a417ec4f None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:10.378893 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04190331-3e12-4135-b459-4797a417ec4f None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:10.379169 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04190331-3e12-4135-b459-4797a417ec4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:10.404768 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:26:10.408027 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-04190331-3e12-4135-b459-4797a417ec4f None None] MFA Rules not processed for user `1728596610fe48f49f459aab2c1eef61`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:26:10.410748 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-04190331-3e12-4135-b459-4797a417ec4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:10.478653 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1210/2438] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:26:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:26:10.491188 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37299620-54c6-45d2-b8b4-89714b8cae7d admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:10.492895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-37299620-54c6-45d2-b8b4-89714b8cae7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:10.553302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37299620-54c6-45d2-b8b4-89714b8cae7d admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:10.556904 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a87fb3-19b8-4581-bee3-195dcac7b823 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:10.557562 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a87fb3-19b8-4581-bee3-195dcac7b823 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:10.557857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a87fb3-19b8-4581-bee3-195dcac7b823 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:10.558121 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a87fb3-19b8-4581-bee3-195dcac7b823 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:10.567093 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a87fb3-19b8-4581-bee3-195dcac7b823 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:10.568518 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a87fb3-19b8-4581-bee3-195dcac7b823 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:10.577976 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1229/2439] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:26:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:26:20.475424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=26 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:26:20.573520 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:26:25.001534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a6b02-aa34-4338-ac53-4ba80f1d4f43 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:25.002034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a6b02-aa34-4338-ac53-4ba80f1d4f43 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:25.002150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4a6b02-aa34-4338-ac53-4ba80f1d4f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:25.021882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-fd4a6b02-aa34-4338-ac53-4ba80f1d4f43 None None] MFA Rules not processed for user `8429a502bca744fc9f5d5701fe9c766c`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:26:25.023089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-fd4a6b02-aa34-4338-ac53-4ba80f1d4f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:25.079066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1211/2440] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:26:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:26:35.072257 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=25 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:26:35.234919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a87fb3-19b8-4581-bee3-195dcac7b823 service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:35.309659 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a87fb3-19b8-4581-bee3-195dcac7b823 service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:35.314923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e599d-b651-4b58-b5fa-dbb389b67822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:35.316905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e599d-b651-4b58-b5fa-dbb389b67822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:35.317336 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e599d-b651-4b58-b5fa-dbb389b67822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:35.317336 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e599d-b651-4b58-b5fa-dbb389b67822 admin admin] PATH_INFO: `/v3/users/1728596610fe48f49f459aab2c1eef61` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:35.318753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e599d-b651-4b58-b5fa-dbb389b67822 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1728596610fe48f49f459aab2c1eef61)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:35.321477 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e599d-b651-4b58-b5fa-dbb389b67822 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:35.385709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8e8e599d-b651-4b58-b5fa-dbb389b67822 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1728596610fe48f49f459aab2c1eef61'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:35.386847 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8e8e599d-b651-4b58-b5fa-dbb389b67822 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1728596610fe48f49f459aab2c1eef61'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:35.396205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8e8e599d-b651-4b58-b5fa-dbb389b67822 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1728596610fe48f49f459aab2c1eef61'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:35.397002 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8e8e599d-b651-4b58-b5fa-dbb389b67822 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1728596610fe48f49f459aab2c1eef61'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:35.405733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8e8e599d-b651-4b58-b5fa-dbb389b67822 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1728596610fe48f49f459aab2c1eef61'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:35.427429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1230/2441] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:26:35 2022] DELETE /identity/v3/users/1728596610fe48f49f459aab2c1eef61 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:26:35.432052 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bef14c-ce47-4b39-baa4-68eef44382dc None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:35.432347 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bef14c-ce47-4b39-baa4-68eef44382dc None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:35.432633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bef14c-ce47-4b39-baa4-68eef44382dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:35.457426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-f0bef14c-ce47-4b39-baa4-68eef44382dc None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:26:35.458818 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f0bef14c-ce47-4b39-baa4-68eef44382dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:35.553678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1212/2442] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:26:35.563435 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e599d-b651-4b58-b5fa-dbb389b67822 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:35.564958 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-8e8e599d-b651-4b58-b5fa-dbb389b67822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:35.623243 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e599d-b651-4b58-b5fa-dbb389b67822 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:35.626903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b83825-f02b-4c28-aa95-58d152d8d1ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:35.627514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b83825-f02b-4c28-aa95-58d152d8d1ed service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:35.627796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b83825-f02b-4c28-aa95-58d152d8d1ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:35.628072 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b83825-f02b-4c28-aa95-58d152d8d1ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:35.636867 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b83825-f02b-4c28-aa95-58d152d8d1ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:35.638325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b83825-f02b-4c28-aa95-58d152d8d1ed service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:35.647820 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1231/2443] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:26:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:26:35.927030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bef14c-ce47-4b39-baa4-68eef44382dc None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:35.928794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f0bef14c-ce47-4b39-baa4-68eef44382dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:35.950624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bef14c-ce47-4b39-baa4-68eef44382dc None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:35.955532 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:35.956347 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:35.956667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:35.956932 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8 admin admin] PATH_INFO: `/v3/projects/4a803e0aefca45b3b0756c3d451c4473` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:35.958184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a803e0aefca45b3b0756c3d451c4473)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:35.960238 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:36.022028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a803e0aefca45b3b0756c3d451c4473', 'request_id': 'req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:36.028532 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a803e0aefca45b3b0756c3d451c4473', 'request_id': 'req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:36.039774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a803e0aefca45b3b0756c3d451c4473', 'request_id': 'req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:36.051038 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8 admin admin] The token cache is being invalidate because project 4a803e0aefca45b3b0756c3d451c4473 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:26:36.051337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:36.052000 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:26:36.053515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1213/2444] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:26:35 2022] DELETE /identity/v3/projects/4a803e0aefca45b3b0756c3d451c4473 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:26:45.643483 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:26:46.047424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:26:48.037446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d1f3e-515c-4c52-b476-54e323594156 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:48.037975 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d1f3e-515c-4c52-b476-54e323594156 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:48.038087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d1f3e-515c-4c52-b476-54e323594156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:48.058914 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-658d1f3e-515c-4c52-b476-54e323594156 None None] MFA Rules not processed for user `bc69cf267bea41bf890bc7551d84ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:26:48.060127 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-658d1f3e-515c-4c52-b476-54e323594156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:48.116985 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1232/2445] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 12:26:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:26:48.130983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:26:48.132384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:26:48.228364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2d5da-c5d9-44df-b69d-50a92c1d74e8 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:26:48.234248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8b505-c71d-466b-a228-ecd95f53a10a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975a8d7349994520bcbfb5f75e52aac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:26:48.234958 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8b505-c71d-466b-a228-ecd95f53a10a service swift] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:26:48.235248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8b505-c71d-466b-a228-ecd95f53a10a service swift] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:26:48.235517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8b505-c71d-466b-a228-ecd95f53a10a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:26:48.245049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8b505-c71d-466b-a228-ecd95f53a10a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:26:48.246682 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8b505-c71d-466b-a228-ecd95f53a10a service swift] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:26:48.254928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1214/2446] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 12:26:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 12:26:58.108823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=25 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:26:58.251555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=32 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:27:04.555890 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b61e0-16b3-4e03-8e86-fa253d9a67c5 None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:27:04.556452 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b61e0-16b3-4e03-8e86-fa253d9a67c5 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:27:04.556452 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b61e0-16b3-4e03-8e86-fa253d9a67c5 None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:27:04.557807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1233/2447] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 12:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:27:04.563691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3b930-9d07-4008-b4e9-ab9bb349cfcc None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:27:04.564153 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3b930-9d07-4008-b4e9-ab9bb349cfcc None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:27:04.564278 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3b930-9d07-4008-b4e9-ab9bb349cfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:27:04.585445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-70a3b930-9d07-4008-b4e9-ab9bb349cfcc None None] MFA Rules not processed for user `982b82e8ac424318a934f9149fb91706`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:27:04.586580 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-70a3b930-9d07-4008-b4e9-ab9bb349cfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:27:04.656370 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1215/2448] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 12:27:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:27:04.668566 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b61e0-16b3-4e03-8e86-fa253d9a67c5 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:27:04.670152 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b81b61e0-16b3-4e03-8e86-fa253d9a67c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:27:04.728670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b61e0-16b3-4e03-8e86-fa253d9a67c5 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:27:04.732202 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27653c-4f71-42f1-a2d2-b0473260ae7e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e08e00bcb5f47ffab58974ba376938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:27:04.732801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27653c-4f71-42f1-a2d2-b0473260ae7e service glance] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:27:04.733080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27653c-4f71-42f1-a2d2-b0473260ae7e service glance] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:27:04.733397 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27653c-4f71-42f1-a2d2-b0473260ae7e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:27:04.743358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27653c-4f71-42f1-a2d2-b0473260ae7e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:27:04.744771 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27653c-4f71-42f1-a2d2-b0473260ae7e service glance] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:27:04.755450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1234/2449] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:27:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:27:04.824750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e74f1-4fcf-4cb2-acf3-4dfaf7d38ecf None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:27:04.825033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e74f1-4fcf-4cb2-acf3-4dfaf7d38ecf None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:27:04.825334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e74f1-4fcf-4cb2-acf3-4dfaf7d38ecf None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:27:04.826681 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1216/2450] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 12:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:27:04.832134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390c317-e689-4984-b5b1-4d81b7ac5c91 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:27:04.832422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390c317-e689-4984-b5b1-4d81b7ac5c91 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:27:04.832699 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390c317-e689-4984-b5b1-4d81b7ac5c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:27:04.853294 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-a390c317-e689-4984-b5b1-4d81b7ac5c91 None None] MFA Rules not processed for user `ba75586d3a7d4278ad6c0c31a1bed3df`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:27:04.854458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a390c317-e689-4984-b5b1-4d81b7ac5c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:27:04.905082 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1235/2451] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 12:27:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:27:04.918004 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e74f1-4fcf-4cb2-acf3-4dfaf7d38ecf None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:27:04.919702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-887e74f1-4fcf-4cb2-acf3-4dfaf7d38ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:27:04.932934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e74f1-4fcf-4cb2-acf3-4dfaf7d38ecf None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:27:04.936475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792343f-a652-4440-bd6c-ae631b78163a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:27:04.937066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792343f-a652-4440-bd6c-ae631b78163a service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:27:04.937373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792343f-a652-4440-bd6c-ae631b78163a service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:27:04.937643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792343f-a652-4440-bd6c-ae631b78163a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:27:04.945178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792343f-a652-4440-bd6c-ae631b78163a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:27:04.946579 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792343f-a652-4440-bd6c-ae631b78163a service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:27:04.954277 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1217/2452] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:27:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:27:05.035688 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c3f25-a19a-4f72-a1a8-f683b18c94bc None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:27:05.035969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c3f25-a19a-4f72-a1a8-f683b18c94bc None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:27:05.036240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c3f25-a19a-4f72-a1a8-f683b18c94bc None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:27:05.037636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1236/2453] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 12:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:27:05.042570 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72840a9c-a8a5-4aef-8113-abf0acd87f27 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:27:05.042860 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72840a9c-a8a5-4aef-8113-abf0acd87f27 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:27:05.043161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72840a9c-a8a5-4aef-8113-abf0acd87f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:27:05.062921 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-72840a9c-a8a5-4aef-8113-abf0acd87f27 None None] MFA Rules not processed for user `982b82e8ac424318a934f9149fb91706`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:27:05.064128 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-72840a9c-a8a5-4aef-8113-abf0acd87f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:27:05.074117 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1218/2454] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 12:27:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:27:05.085439 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c3f25-a19a-4f72-a1a8-f683b18c94bc None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:27:05.087106 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-551c3f25-a19a-4f72-a1a8-f683b18c94bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:27:05.100509 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c3f25-a19a-4f72-a1a8-f683b18c94bc None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:27:05.104087 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946eed02-5d26-42ae-9e5b-e7f3c3685d53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:27:05.104615 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eed02-5d26-42ae-9e5b-e7f3c3685d53 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:27:05.104890 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eed02-5d26-42ae-9e5b-e7f3c3685d53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:27:05.105185 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946eed02-5d26-42ae-9e5b-e7f3c3685d53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:27:05.115116 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946eed02-5d26-42ae-9e5b-e7f3c3685d53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:27:05.116528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946eed02-5d26-42ae-9e5b-e7f3c3685d53 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:27:05.127006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1237/2455] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:27:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:27:09.504307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3250d81-8ce7-42db-b76d-e45197baf5f7 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:27:09.504841 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3250d81-8ce7-42db-b76d-e45197baf5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:27:09.504841 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3250d81-8ce7-42db-b76d-e45197baf5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:27:09.524235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e3250d81-8ce7-42db-b76d-e45197baf5f7 None None] MFA Rules not processed for user `cc96808ff3fb41f085f09aa943c3d016`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:27:09.525548 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e3250d81-8ce7-42db-b76d-e45197baf5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:27:09.587383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1219/2456] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:27:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:27:10.290881 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f69d3-8e41-4bba-a906-a0fc3a1b30bb None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:27:10.291362 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f69d3-8e41-4bba-a906-a0fc3a1b30bb None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:27:10.291464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f69d3-8e41-4bba-a906-a0fc3a1b30bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:27:10.310752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-ef8f69d3-8e41-4bba-a906-a0fc3a1b30bb None None] MFA Rules not processed for user `bc69cf267bea41bf890bc7551d84ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:27:10.311976 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-ef8f69d3-8e41-4bba-a906-a0fc3a1b30bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:27:10.319934 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1238/2457] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 12:27:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:27:10.343587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3250d81-8ce7-42db-b76d-e45197baf5f7 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:27:10.356208 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3250d81-8ce7-42db-b76d-e45197baf5f7 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:27:10.361288 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ce13e-e492-4d18-88ec-410b3c229676 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975a8d7349994520bcbfb5f75e52aac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:27:10.361892 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ce13e-e492-4d18-88ec-410b3c229676 service swift] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:27:10.362275 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ce13e-e492-4d18-88ec-410b3c229676 service swift] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:27:10.363057 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ce13e-e492-4d18-88ec-410b3c229676 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:27:10.371507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ce13e-e492-4d18-88ec-410b3c229676 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:27:10.373168 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ce13e-e492-4d18-88ec-410b3c229676 service swift] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:27:10.381771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1220/2458] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 12:27:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 12:27:20.305551 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=58 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:27:20.378053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=71 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:27:41.608706 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f69d3-8e41-4bba-a906-a0fc3a1b30bb None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:27:41.618763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f69d3-8e41-4bba-a906-a0fc3a1b30bb None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:27:41.622300 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65b04c-1201-4eb7-9374-3df4e3f6ca2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:27:41.622892 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65b04c-1201-4eb7-9374-3df4e3f6ca2d service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:27:41.623168 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65b04c-1201-4eb7-9374-3df4e3f6ca2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:27:41.623437 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65b04c-1201-4eb7-9374-3df4e3f6ca2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:27:41.625181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-ef65b04c-1201-4eb7-9374-3df4e3f6ca2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:27:41.638110 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65b04c-1201-4eb7-9374-3df4e3f6ca2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:27:41.639489 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65b04c-1201-4eb7-9374-3df4e3f6ca2d service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:27:41.650100 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1239/2459] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:27:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:27:42.113878 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc31ea-4867-4fe5-922e-d0781498d8a0 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:27:42.114400 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc31ea-4867-4fe5-922e-d0781498d8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:27:42.114400 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc31ea-4867-4fe5-922e-d0781498d8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:27:42.133602 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-1efc31ea-4867-4fe5-922e-d0781498d8a0 None None] MFA Rules not processed for user `bc69cf267bea41bf890bc7551d84ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:27:42.134808 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1efc31ea-4867-4fe5-922e-d0781498d8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:27:42.141936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1221/2460] 10.222.0.60 () {58 vars in 967 bytes} [Wed Sep 28 12:27:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:27:42.154675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65b04c-1201-4eb7-9374-3df4e3f6ca2d service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:27:42.166748 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65b04c-1201-4eb7-9374-3df4e3f6ca2d service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:27:42.171773 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1c39c-8805-46e3-9ac4-f03183a844e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975a8d7349994520bcbfb5f75e52aac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:27:42.172365 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1c39c-8805-46e3-9ac4-f03183a844e6 service swift] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:27:42.172645 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1c39c-8805-46e3-9ac4-f03183a844e6 service swift] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:27:42.172930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e1c39c-8805-46e3-9ac4-f03183a844e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:27:42.181512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1c39c-8805-46e3-9ac4-f03183a844e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:27:42.183078 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e1c39c-8805-46e3-9ac4-f03183a844e6 service swift] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:27:42.191565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1240/2461] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 12:27:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 12:27:42.836202 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc31ea-4867-4fe5-922e-d0781498d8a0 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:27:42.845820 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc31ea-4867-4fe5-922e-d0781498d8a0 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:27:42.849426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9aa6f-59e8-41de-a575-0a690b2c0a68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:27:42.850041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9aa6f-59e8-41de-a575-0a690b2c0a68 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:27:42.850316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9aa6f-59e8-41de-a575-0a690b2c0a68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:27:42.850665 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9aa6f-59e8-41de-a575-0a690b2c0a68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:27:42.852366 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-68f9aa6f-59e8-41de-a575-0a690b2c0a68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:27:42.865611 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9aa6f-59e8-41de-a575-0a690b2c0a68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:27:42.867598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9aa6f-59e8-41de-a575-0a690b2c0a68 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:27:42.878135 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1222/2462] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:27:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:27:52.188152 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=12 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:27:52.873517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=9 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:28:28.549755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1c39c-8805-46e3-9ac4-f03183a844e6 service swift] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:28:28.551341 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d5e1c39c-8805-46e3-9ac4-f03183a844e6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:28:28.566760 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e1c39c-8805-46e3-9ac4-f03183a844e6 service swift] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:28:28.572853 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cfb434-beb4-4001-999b-9dc8f995917f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:28:28.573478 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfb434-beb4-4001-999b-9dc8f995917f service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:28:28.573755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfb434-beb4-4001-999b-9dc8f995917f service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:28:28.574022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfb434-beb4-4001-999b-9dc8f995917f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:28:28.575673 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-75cfb434-beb4-4001-999b-9dc8f995917f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:28:28.632075 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cfb434-beb4-4001-999b-9dc8f995917f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:28:28.634271 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cfb434-beb4-4001-999b-9dc8f995917f service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:28:28.643292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1241/2463] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:28:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:28:31.953971 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9aa6f-59e8-41de-a575-0a690b2c0a68 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:28:31.971286 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9aa6f-59e8-41de-a575-0a690b2c0a68 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:28:31.977018 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87709e71-ac46-4483-aab0-37275c905be7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:28:31.977656 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87709e71-ac46-4483-aab0-37275c905be7 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:28:31.977936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87709e71-ac46-4483-aab0-37275c905be7 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:28:31.978210 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87709e71-ac46-4483-aab0-37275c905be7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:28:31.979952 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-87709e71-ac46-4483-aab0-37275c905be7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:28:31.993164 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87709e71-ac46-4483-aab0-37275c905be7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:28:31.994766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87709e71-ac46-4483-aab0-37275c905be7 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:28:32.016066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1223/2464] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:28:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:28:38.639114 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=28 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:28:42.001331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=6 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:28:43.395594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f9581-0b11-42e6-a998-ce6e3ba58054 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:28:43.396079 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f9581-0b11-42e6-a998-ce6e3ba58054 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:28:43.396198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f9581-0b11-42e6-a998-ce6e3ba58054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:28:43.416336 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-e87f9581-0b11-42e6-a998-ce6e3ba58054 None None] MFA Rules not processed for user `8429a502bca744fc9f5d5701fe9c766c`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:28:43.417602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e87f9581-0b11-42e6-a998-ce6e3ba58054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:28:43.424380 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1242/2465] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:28:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:28:43.718177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87709e71-ac46-4483-aab0-37275c905be7 service nova] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:28:43.728613 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87709e71-ac46-4483-aab0-37275c905be7 service nova] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:28:43.732406 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1371ce-2dc7-4f5c-9e87-c5ee410e81b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:28:43.733013 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1371ce-2dc7-4f5c-9e87-c5ee410e81b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:28:43.733378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1371ce-2dc7-4f5c-9e87-c5ee410e81b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:28:43.733656 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1371ce-2dc7-4f5c-9e87-c5ee410e81b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:28:43.735452 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-fe1371ce-2dc7-4f5c-9e87-c5ee410e81b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:28:43.746321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1371ce-2dc7-4f5c-9e87-c5ee410e81b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:28:43.747638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1371ce-2dc7-4f5c-9e87-c5ee410e81b2 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:28:43.756752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1224/2466] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:28:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:28:44.073224 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f9581-0b11-42e6-a998-ce6e3ba58054 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:28:44.074921 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e87f9581-0b11-42e6-a998-ce6e3ba58054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:28:44.145247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f9581-0b11-42e6-a998-ce6e3ba58054 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:28:44.150790 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0beaf-a7d6-4416-952b-ee0d8ab638bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:28:44.151393 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0beaf-a7d6-4416-952b-ee0d8ab638bf service placement] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:28:44.151669 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0beaf-a7d6-4416-952b-ee0d8ab638bf service placement] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:28:44.151938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0beaf-a7d6-4416-952b-ee0d8ab638bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:28:44.153736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-4bb0beaf-a7d6-4416-952b-ee0d8ab638bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:28:44.165591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0beaf-a7d6-4416-952b-ee0d8ab638bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:28:44.167176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0beaf-a7d6-4416-952b-ee0d8ab638bf service placement] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:28:44.177153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1243/2467] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:28:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:28:46.567745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1371ce-2dc7-4f5c-9e87-c5ee410e81b2 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:28:46.577214 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1371ce-2dc7-4f5c-9e87-c5ee410e81b2 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:28:46.580825 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54e417-9ddf-4823-b867-9ec6aebc9790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:28:46.581517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54e417-9ddf-4823-b867-9ec6aebc9790 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:28:46.581799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54e417-9ddf-4823-b867-9ec6aebc9790 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:28:46.582074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54e417-9ddf-4823-b867-9ec6aebc9790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:28:46.583763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-af54e417-9ddf-4823-b867-9ec6aebc9790 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:28:46.593223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54e417-9ddf-4823-b867-9ec6aebc9790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:28:46.594647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54e417-9ddf-4823-b867-9ec6aebc9790 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:28:46.602821 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1225/2468] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:28:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:28:52.478459 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0beaf-a7d6-4416-952b-ee0d8ab638bf service placement] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:28:52.491662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0beaf-a7d6-4416-952b-ee0d8ab638bf service placement] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:28:52.497591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08d108-aa13-4844-ba08-5fbf0e6831fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:28:52.498618 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08d108-aa13-4844-ba08-5fbf0e6831fa service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:28:52.498897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08d108-aa13-4844-ba08-5fbf0e6831fa service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:28:52.499168 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08d108-aa13-4844-ba08-5fbf0e6831fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:28:52.501321 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-da08d108-aa13-4844-ba08-5fbf0e6831fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:28:52.514153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08d108-aa13-4844-ba08-5fbf0e6831fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:28:52.515754 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08d108-aa13-4844-ba08-5fbf0e6831fa service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:28:52.528658 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1244/2469] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:28:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:28:56.599210 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=12 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:28:57.494578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f5da2-daa9-4924-aee5-e218ea6d5661 None None] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:28:57.494867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f5da2-daa9-4924-aee5-e218ea6d5661 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:28:57.495425 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f5da2-daa9-4924-aee5-e218ea6d5661 None None] PATH_INFO: `/` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:28:57.497471 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1226/2470] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 12:28:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:28:57.503897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c2fcd-d94f-4712-b7e1-c6f985fcf5eb None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:28:57.504428 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c2fcd-d94f-4712-b7e1-c6f985fcf5eb None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:28:57.504428 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c2fcd-d94f-4712-b7e1-c6f985fcf5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:28:57.526881 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-3a8c2fcd-d94f-4712-b7e1-c6f985fcf5eb None None] MFA Rules not processed for user `ba75586d3a7d4278ad6c0c31a1bed3df`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:28:57.527987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-3a8c2fcd-d94f-4712-b7e1-c6f985fcf5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:28:57.537330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1245/2471] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 12:28:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:28:57.560194 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f5da2-daa9-4924-aee5-e218ea6d5661 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:28:57.572779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f5da2-daa9-4924-aee5-e218ea6d5661 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:28:57.577125 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb06a8-5639-4c8f-aa21-ca37f1592a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:28:57.577758 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb06a8-5639-4c8f-aa21-ca37f1592a0c service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:28:57.578043 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb06a8-5639-4c8f-aa21-ca37f1592a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:28:57.578320 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb06a8-5639-4c8f-aa21-ca37f1592a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:28:57.585829 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb06a8-5639-4c8f-aa21-ca37f1592a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:28:57.588197 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb06a8-5639-4c8f-aa21-ca37f1592a0c service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:28:57.596789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1227/2472] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:28:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:28:57.754575 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e346ba-175e-4362-ab98-dd25b99dc19b None None] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:28:57.754958 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e346ba-175e-4362-ab98-dd25b99dc19b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:28:57.755301 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e346ba-175e-4362-ab98-dd25b99dc19b None None] PATH_INFO: `/` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:28:57.756730 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1246/2473] 10.222.0.60 () {54 vars in 921 bytes} [Wed Sep 28 12:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 12:28:57.762412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a004a-a372-4e8a-9fcf-ddf727767a2a None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:28:57.762920 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a004a-a372-4e8a-9fcf-ddf727767a2a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:28:57.763028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a004a-a372-4e8a-9fcf-ddf727767a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:28:57.783819 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-232a004a-a372-4e8a-9fcf-ddf727767a2a None None] MFA Rules not processed for user `982b82e8ac424318a934f9149fb91706`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:28:57.784937 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-232a004a-a372-4e8a-9fcf-ddf727767a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:28:57.794819 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1228/2474] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Sep 28 12:28:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:28:57.805255 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e346ba-175e-4362-ab98-dd25b99dc19b None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:28:57.816256 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e346ba-175e-4362-ab98-dd25b99dc19b None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:28:57.820031 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68c620-0259-42e0-89bd-473c4534fd90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:28:57.820626 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68c620-0259-42e0-89bd-473c4534fd90 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:28:57.820977 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68c620-0259-42e0-89bd-473c4534fd90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:28:57.821323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68c620-0259-42e0-89bd-473c4534fd90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:28:57.831861 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68c620-0259-42e0-89bd-473c4534fd90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:28:57.833384 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68c620-0259-42e0-89bd-473c4534fd90 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:28:57.845702 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1247/2475] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:28:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:28:59.629598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a004a-a372-4e8a-9fcf-ddf727767a2a None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:28:59.631079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-232a004a-a372-4e8a-9fcf-ddf727767a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:28:59.644648 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a004a-a372-4e8a-9fcf-ddf727767a2a None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:28:59.650404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173699f2-0565-4690-bc3c-f35105282e39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:28:59.650995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173699f2-0565-4690-bc3c-f35105282e39 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:28:59.651276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173699f2-0565-4690-bc3c-f35105282e39 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:28:59.651551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173699f2-0565-4690-bc3c-f35105282e39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:28:59.653220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-173699f2-0565-4690-bc3c-f35105282e39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:28:59.710235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173699f2-0565-4690-bc3c-f35105282e39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:28:59.712329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173699f2-0565-4690-bc3c-f35105282e39 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:28:59.721658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1229/2476] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:28:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:29:05.356323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68c620-0259-42e0-89bd-473c4534fd90 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:29:05.369588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68c620-0259-42e0-89bd-473c4534fd90 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:29:05.375895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb59ce8-3c4b-479f-be55-eb467b72485f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:29:05.376517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb59ce8-3c4b-479f-be55-eb467b72485f service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:29:05.376801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb59ce8-3c4b-479f-be55-eb467b72485f service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:29:05.377078 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb59ce8-3c4b-479f-be55-eb467b72485f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:29:05.379112 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-efb59ce8-3c4b-479f-be55-eb467b72485f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:29:05.393822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb59ce8-3c4b-479f-be55-eb467b72485f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:29:05.395400 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb59ce8-3c4b-479f-be55-eb467b72485f service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:29:05.407541 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1248/2477] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:29:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:29:09.329590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a377ce9-9c6d-4349-9de2-447ad59a17a7 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:29:09.330111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a377ce9-9c6d-4349-9de2-447ad59a17a7 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:29:09.330111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a377ce9-9c6d-4349-9de2-447ad59a17a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:29:09.350904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-6a377ce9-9c6d-4349-9de2-447ad59a17a7 None None] MFA Rules not processed for user `4d7b7c68a699443aa5c0859094c265ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:29:09.352221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-6a377ce9-9c6d-4349-9de2-447ad59a17a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:29:09.359308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1230/2478] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:29:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:29:09.670469 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb59ce8-3c4b-479f-be55-eb467b72485f service nova] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:29:09.674722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=57 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:29:09.681286 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb59ce8-3c4b-479f-be55-eb467b72485f service nova] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:29:09.684741 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d83958-32d7-424b-8258-50987c54a45f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:29:09.685363 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d83958-32d7-424b-8258-50987c54a45f service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:29:09.685641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d83958-32d7-424b-8258-50987c54a45f service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:29:09.685908 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d83958-32d7-424b-8258-50987c54a45f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:29:09.687585 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-f8d83958-32d7-424b-8258-50987c54a45f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:29:09.697355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d83958-32d7-424b-8258-50987c54a45f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:29:09.698738 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d83958-32d7-424b-8258-50987c54a45f service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:29:09.706721 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1249/2479] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:29:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:29:19.343977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=40 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:29:19.703160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=5 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:29:27.697309 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13d50c-f75a-4833-ac09-d5b6dd3fb0d2 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:29:27.697848 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13d50c-f75a-4833-ac09-d5b6dd3fb0d2 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:29:27.698040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13d50c-f75a-4833-ac09-d5b6dd3fb0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:29:27.724046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-cb13d50c-f75a-4833-ac09-d5b6dd3fb0d2 None None] MFA Rules not processed for user `bc69cf267bea41bf890bc7551d84ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:29:27.725268 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-cb13d50c-f75a-4833-ac09-d5b6dd3fb0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:29:27.740374 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1231/2480] 10.222.0.60 () {58 vars in 967 bytes} [Wed Sep 28 12:29:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:29:27.807601 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d83958-32d7-424b-8258-50987c54a45f service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:29:27.819570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d83958-32d7-424b-8258-50987c54a45f service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:29:27.824506 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641cada-9f79-4c83-84cb-563438500705 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975a8d7349994520bcbfb5f75e52aac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:29:27.825100 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641cada-9f79-4c83-84cb-563438500705 service swift] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:29:27.825487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641cada-9f79-4c83-84cb-563438500705 service swift] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:29:27.826536 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641cada-9f79-4c83-84cb-563438500705 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:29:27.836341 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641cada-9f79-4c83-84cb-563438500705 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:29:27.837988 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641cada-9f79-4c83-84cb-563438500705 service swift] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:29:27.847306 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1250/2481] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 12:29:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 12:29:28.244067 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13d50c-f75a-4833-ac09-d5b6dd3fb0d2 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:29:28.263511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13d50c-f75a-4833-ac09-d5b6dd3fb0d2 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:29:28.272837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3b292-5388-4f39-9885-5400187b225e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:29:28.272837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3b292-5388-4f39-9885-5400187b225e service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:29:28.273177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3b292-5388-4f39-9885-5400187b225e service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:29:28.273505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3b292-5388-4f39-9885-5400187b225e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:29:28.275524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-3fa3b292-5388-4f39-9885-5400187b225e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:29:28.344962 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3b292-5388-4f39-9885-5400187b225e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:29:28.346749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3b292-5388-4f39-9885-5400187b225e service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:29:28.359759 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1232/2482] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:29:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:29:35.225386 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641cada-9f79-4c83-84cb-563438500705 service swift] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:29:35.226944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5641cada-9f79-4c83-84cb-563438500705 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:29:35.241330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641cada-9f79-4c83-84cb-563438500705 service swift] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:29:35.246474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf061c3-5d69-4a97-a695-7ae7d49cf1e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:29:35.247161 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf061c3-5d69-4a97-a695-7ae7d49cf1e2 service placement] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:29:35.247510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf061c3-5d69-4a97-a695-7ae7d49cf1e2 service placement] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:29:35.247849 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf061c3-5d69-4a97-a695-7ae7d49cf1e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:29:35.249608 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-acf061c3-5d69-4a97-a695-7ae7d49cf1e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:29:35.261488 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf061c3-5d69-4a97-a695-7ae7d49cf1e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:29:35.263139 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf061c3-5d69-4a97-a695-7ae7d49cf1e2 service placement] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:29:35.277725 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1251/2483] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:29:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:29:35.285463 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3b292-5388-4f39-9885-5400187b225e service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:29:35.287444 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-3fa3b292-5388-4f39-9885-5400187b225e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:29:35.302405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3b292-5388-4f39-9885-5400187b225e service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:29:35.308753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb0c4e-17fd-424c-8558-8449d3d478c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:29:35.309440 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb0c4e-17fd-424c-8558-8449d3d478c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:29:35.309822 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb0c4e-17fd-424c-8558-8449d3d478c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:29:35.310120 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb0c4e-17fd-424c-8558-8449d3d478c7 admin admin] PATH_INFO: `/v3/users/2ab18214df00412bb7440535ac725056` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:29:35.311254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb0c4e-17fd-424c-8558-8449d3d478c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ab18214df00412bb7440535ac725056)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:29:35.313175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb0c4e-17fd-424c-8558-8449d3d478c7 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:29:35.360162 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-c1eb0c4e-17fd-424c-8558-8449d3d478c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ab18214df00412bb7440535ac725056'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:35.360850 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-c1eb0c4e-17fd-424c-8558-8449d3d478c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ab18214df00412bb7440535ac725056'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:35.373339 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-c1eb0c4e-17fd-424c-8558-8449d3d478c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ab18214df00412bb7440535ac725056'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:35.380404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-c1eb0c4e-17fd-424c-8558-8449d3d478c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ab18214df00412bb7440535ac725056'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:35.381022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-c1eb0c4e-17fd-424c-8558-8449d3d478c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ab18214df00412bb7440535ac725056'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:35.394071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1233/2484] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:29:35 2022] DELETE /identity/v3/users/2ab18214df00412bb7440535ac725056 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:29:35.564071 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf061c3-5d69-4a97-a695-7ae7d49cf1e2 service placement] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:29:35.579103 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf061c3-5d69-4a97-a695-7ae7d49cf1e2 service placement] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:29:35.634634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:29:35.635950 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:29:35.636244 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:29:35.636552 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4 admin admin] PATH_INFO: `/v3/projects/19f9f5faf55a4b0cb17bf36d8cb838f6` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:29:35.637629 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f9f5faf55a4b0cb17bf36d8cb838f6)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:29:35.639496 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:29:35.684861 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f9f5faf55a4b0cb17bf36d8cb838f6', 'request_id': 'req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:35.692342 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f9f5faf55a4b0cb17bf36d8cb838f6', 'request_id': 'req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:35.699622 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f9f5faf55a4b0cb17bf36d8cb838f6', 'request_id': 'req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:35.709744 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4 admin admin] The token cache is being invalidate because project 19f9f5faf55a4b0cb17bf36d8cb838f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:29:35.710026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:35.710649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:35.712084 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1252/2485] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:29:35 2022] DELETE /identity/v3/projects/19f9f5faf55a4b0cb17bf36d8cb838f6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:29:35.716252 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb0c4e-17fd-424c-8558-8449d3d478c7 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:29:35.717664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c1eb0c4e-17fd-424c-8558-8449d3d478c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:29:35.731880 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb0c4e-17fd-424c-8558-8449d3d478c7 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:29:35.786577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23817dd-329e-42d1-8927-1c053c33427c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:29:35.787199 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23817dd-329e-42d1-8927-1c053c33427c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:29:35.787479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23817dd-329e-42d1-8927-1c053c33427c admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:29:35.787749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23817dd-329e-42d1-8927-1c053c33427c admin admin] PATH_INFO: `/v3/users/1d44cd6ef5824545afafcb831fd1be32` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:29:35.788878 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23817dd-329e-42d1-8927-1c053c33427c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d44cd6ef5824545afafcb831fd1be32)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:29:35.790805 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23817dd-329e-42d1-8927-1c053c33427c admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:29:35.793036 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:29:35.840066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d23817dd-329e-42d1-8927-1c053c33427c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d44cd6ef5824545afafcb831fd1be32'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:35.840637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d23817dd-329e-42d1-8927-1c053c33427c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d44cd6ef5824545afafcb831fd1be32'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:35.852785 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d23817dd-329e-42d1-8927-1c053c33427c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d44cd6ef5824545afafcb831fd1be32'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:35.860692 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d23817dd-329e-42d1-8927-1c053c33427c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d44cd6ef5824545afafcb831fd1be32'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:35.861339 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d23817dd-329e-42d1-8927-1c053c33427c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d44cd6ef5824545afafcb831fd1be32'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:35.874190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1234/2486] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:29:35 2022] DELETE /identity/v3/users/1d44cd6ef5824545afafcb831fd1be32 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:29:36.046041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:29:36.048470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:29:36.061963 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b73911-9cbf-4148-9bdc-7a5135dc38d4 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:29:36.122480 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662af506-9062-4db6-8e5d-88721b4c698e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:29:36.122936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662af506-9062-4db6-8e5d-88721b4c698e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:29:36.123343 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662af506-9062-4db6-8e5d-88721b4c698e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:29:36.123626 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662af506-9062-4db6-8e5d-88721b4c698e admin admin] PATH_INFO: `/v3/projects/6131a9e67e1b4ac2ad3d28f2fad23f67` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:29:36.124783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662af506-9062-4db6-8e5d-88721b4c698e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6131a9e67e1b4ac2ad3d28f2fad23f67)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:29:36.126751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662af506-9062-4db6-8e5d-88721b4c698e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:29:36.133800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:29:36.177940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-662af506-9062-4db6-8e5d-88721b4c698e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6131a9e67e1b4ac2ad3d28f2fad23f67', 'request_id': 'req-662af506-9062-4db6-8e5d-88721b4c698e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:36.184282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-662af506-9062-4db6-8e5d-88721b4c698e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6131a9e67e1b4ac2ad3d28f2fad23f67', 'request_id': 'req-662af506-9062-4db6-8e5d-88721b4c698e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:36.191112 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-662af506-9062-4db6-8e5d-88721b4c698e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6131a9e67e1b4ac2ad3d28f2fad23f67', 'request_id': 'req-662af506-9062-4db6-8e5d-88721b4c698e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:36.201492 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-662af506-9062-4db6-8e5d-88721b4c698e admin admin] The token cache is being invalidate because project 6131a9e67e1b4ac2ad3d28f2fad23f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:29:36.201777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-662af506-9062-4db6-8e5d-88721b4c698e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:36.202431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-662af506-9062-4db6-8e5d-88721b4c698e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:29:36.204198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1253/2487] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:29:36 2022] DELETE /identity/v3/projects/6131a9e67e1b4ac2ad3d28f2fad23f67 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:29:44.191305 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23817dd-329e-42d1-8927-1c053c33427c admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:29:44.192742 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d23817dd-329e-42d1-8927-1c053c33427c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:29:44.265760 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23817dd-329e-42d1-8927-1c053c33427c admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:29:44.328592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcf5aa-6649-4558-be32-25b78cf10064 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:29:44.329248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcf5aa-6649-4558-be32-25b78cf10064 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:29:44.329536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcf5aa-6649-4558-be32-25b78cf10064 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:29:44.329814 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcf5aa-6649-4558-be32-25b78cf10064 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:29:44.331634 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-05bcf5aa-6649-4558-be32-25b78cf10064 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:29:44.345058 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:29:44.391345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcf5aa-6649-4558-be32-25b78cf10064 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:29:44.392921 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcf5aa-6649-4558-be32-25b78cf10064 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:29:44.402316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1235/2488] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:29:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:29:46.198534 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:29:54.397700 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=21 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:29:55.474721 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706068a-1be7-43d7-a155-553db457215b None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:29:55.475265 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706068a-1be7-43d7-a155-553db457215b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:29:55.475380 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706068a-1be7-43d7-a155-553db457215b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:29:55.496107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-9706068a-1be7-43d7-a155-553db457215b None None] MFA Rules not processed for user `cc96808ff3fb41f085f09aa943c3d016`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:29:55.497326 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9706068a-1be7-43d7-a155-553db457215b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:29:55.504101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1254/2489] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:29:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:30:05.490277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:30:44.358680 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdbdf5-139e-4a6f-9adb-179ce2af62f8 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:30:44.359204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdbdf5-139e-4a6f-9adb-179ce2af62f8 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:30:44.359302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdbdf5-139e-4a6f-9adb-179ce2af62f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:30:44.390862 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-8cbdbdf5-139e-4a6f-9adb-179ce2af62f8 None None] MFA Rules not processed for user `8429a502bca744fc9f5d5701fe9c766c`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:30:44.392061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8cbdbdf5-139e-4a6f-9adb-179ce2af62f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:30:44.462015 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1236/2490] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:30:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:30:54.460182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=37 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:30:55.381953 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706068a-1be7-43d7-a155-553db457215b None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:30:55.383435 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9706068a-1be7-43d7-a155-553db457215b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:30:55.477833 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706068a-1be7-43d7-a155-553db457215b None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:30:55.482176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152212f-16f0-4865-86b4-82c74f5a5034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:30:55.482789 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152212f-16f0-4865-86b4-82c74f5a5034 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:30:55.483066 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152212f-16f0-4865-86b4-82c74f5a5034 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:30:55.483339 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152212f-16f0-4865-86b4-82c74f5a5034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:30:55.485049 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b152212f-16f0-4865-86b4-82c74f5a5034 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:30:55.547045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152212f-16f0-4865-86b4-82c74f5a5034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:30:55.548477 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152212f-16f0-4865-86b4-82c74f5a5034 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:30:55.557738 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1255/2491] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:30:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:31:00.841709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdbdf5-139e-4a6f-9adb-179ce2af62f8 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:00.844347 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8cbdbdf5-139e-4a6f-9adb-179ce2af62f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:00.990010 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdbdf5-139e-4a6f-9adb-179ce2af62f8 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:00.995525 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6b8a5-3862-46ae-b261-3cac55677f4e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:00.996982 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6b8a5-3862-46ae-b261-3cac55677f4e service placement] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:00.997302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6b8a5-3862-46ae-b261-3cac55677f4e service placement] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:00.997578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6b8a5-3862-46ae-b261-3cac55677f4e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:01.000090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0fd6b8a5-3862-46ae-b261-3cac55677f4e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:01.014053 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6b8a5-3862-46ae-b261-3cac55677f4e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:01.015239 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6b8a5-3862-46ae-b261-3cac55677f4e service placement] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:01.026607 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1237/2492] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:31:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:31:02.580591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152212f-16f0-4865-86b4-82c74f5a5034 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:02.582023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b152212f-16f0-4865-86b4-82c74f5a5034 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:02.614646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152212f-16f0-4865-86b4-82c74f5a5034 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:02.618227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00879a9d-7b27-42d5-85a7-88c0b9eb2e53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:02.618826 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00879a9d-7b27-42d5-85a7-88c0b9eb2e53 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:02.619103 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00879a9d-7b27-42d5-85a7-88c0b9eb2e53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:02.619372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00879a9d-7b27-42d5-85a7-88c0b9eb2e53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:02.621096 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-00879a9d-7b27-42d5-85a7-88c0b9eb2e53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:02.686640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:31:02.705223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00879a9d-7b27-42d5-85a7-88c0b9eb2e53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:02.706647 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00879a9d-7b27-42d5-85a7-88c0b9eb2e53 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:02.725790 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1256/2493] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:31:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:31:10.422549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6b8a5-3862-46ae-b261-3cac55677f4e service placement] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:10.425011 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0fd6b8a5-3862-46ae-b261-3cac55677f4e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:10.445368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6b8a5-3862-46ae-b261-3cac55677f4e service placement] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:10.452577 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ba93f-6af7-43dc-bf84-fc3afd3da617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:10.453350 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ba93f-6af7-43dc-bf84-fc3afd3da617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:10.453632 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ba93f-6af7-43dc-bf84-fc3afd3da617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:10.453900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ba93f-6af7-43dc-bf84-fc3afd3da617 admin admin] PATH_INFO: `/v3/users/49280c781ba745aa83bf4c5db9359973` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:10.455087 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ba93f-6af7-43dc-bf84-fc3afd3da617 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49280c781ba745aa83bf4c5db9359973)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:10.457255 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ba93f-6af7-43dc-bf84-fc3afd3da617 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:10.517801 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-fd6ba93f-6af7-43dc-bf84-fc3afd3da617 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49280c781ba745aa83bf4c5db9359973'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:10.518581 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-fd6ba93f-6af7-43dc-bf84-fc3afd3da617 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49280c781ba745aa83bf4c5db9359973'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:10.531173 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-fd6ba93f-6af7-43dc-bf84-fc3afd3da617 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49280c781ba745aa83bf4c5db9359973'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:10.538806 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-fd6ba93f-6af7-43dc-bf84-fc3afd3da617 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49280c781ba745aa83bf4c5db9359973'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:10.539426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-fd6ba93f-6af7-43dc-bf84-fc3afd3da617 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49280c781ba745aa83bf4c5db9359973'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:10.552476 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1238/2494] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:31:10 2022] DELETE /identity/v3/users/49280c781ba745aa83bf4c5db9359973 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:31:10.729171 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00879a9d-7b27-42d5-85a7-88c0b9eb2e53 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:10.741026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00879a9d-7b27-42d5-85a7-88c0b9eb2e53 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:10.809503 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9816d4-31d8-477a-94d1-346368cfd65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:10.810187 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9816d4-31d8-477a-94d1-346368cfd65e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:10.810538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9816d4-31d8-477a-94d1-346368cfd65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:10.810886 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9816d4-31d8-477a-94d1-346368cfd65e admin admin] PATH_INFO: `/v3/projects/1f563e7675ef4d758e09e1a78b891c9d` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:10.812204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9816d4-31d8-477a-94d1-346368cfd65e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f563e7675ef4d758e09e1a78b891c9d)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:10.814225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9816d4-31d8-477a-94d1-346368cfd65e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:10.861360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-ea9816d4-31d8-477a-94d1-346368cfd65e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f563e7675ef4d758e09e1a78b891c9d', 'request_id': 'req-ea9816d4-31d8-477a-94d1-346368cfd65e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:10.867374 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-ea9816d4-31d8-477a-94d1-346368cfd65e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f563e7675ef4d758e09e1a78b891c9d', 'request_id': 'req-ea9816d4-31d8-477a-94d1-346368cfd65e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:10.874317 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-ea9816d4-31d8-477a-94d1-346368cfd65e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f563e7675ef4d758e09e1a78b891c9d', 'request_id': 'req-ea9816d4-31d8-477a-94d1-346368cfd65e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:10.884816 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-ea9816d4-31d8-477a-94d1-346368cfd65e admin admin] The token cache is being invalidate because project 1f563e7675ef4d758e09e1a78b891c9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:31:10.885111 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-ea9816d4-31d8-477a-94d1-346368cfd65e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:10.885783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-ea9816d4-31d8-477a-94d1-346368cfd65e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:10.888253 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1257/2495] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:31:10 2022] DELETE /identity/v3/projects/1f563e7675ef4d758e09e1a78b891c9d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:31:10.892295 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ba93f-6af7-43dc-bf84-fc3afd3da617 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:10.893739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-fd6ba93f-6af7-43dc-bf84-fc3afd3da617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:10.907452 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ba93f-6af7-43dc-bf84-fc3afd3da617 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:10.940643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:31:10.967714 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070fd2b-46aa-4781-901f-59f822008e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:10.968350 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070fd2b-46aa-4781-901f-59f822008e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:10.968897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070fd2b-46aa-4781-901f-59f822008e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:10.969198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070fd2b-46aa-4781-901f-59f822008e6f admin admin] PATH_INFO: `/v3/users/93fb252cab314529a97fb513e8fa28b5` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:10.970321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f070fd2b-46aa-4781-901f-59f822008e6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=93fb252cab314529a97fb513e8fa28b5)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:10.972211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f070fd2b-46aa-4781-901f-59f822008e6f admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:11.015840 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f070fd2b-46aa-4781-901f-59f822008e6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93fb252cab314529a97fb513e8fa28b5'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:11.016478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f070fd2b-46aa-4781-901f-59f822008e6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93fb252cab314529a97fb513e8fa28b5'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:11.028991 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f070fd2b-46aa-4781-901f-59f822008e6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93fb252cab314529a97fb513e8fa28b5'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:11.036323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f070fd2b-46aa-4781-901f-59f822008e6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93fb252cab314529a97fb513e8fa28b5'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:11.037019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f070fd2b-46aa-4781-901f-59f822008e6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93fb252cab314529a97fb513e8fa28b5'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:11.049637 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1239/2496] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:31:10 2022] DELETE /identity/v3/users/93fb252cab314529a97fb513e8fa28b5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:31:11.227131 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9816d4-31d8-477a-94d1-346368cfd65e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:11.228656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-ea9816d4-31d8-477a-94d1-346368cfd65e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:11.243309 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9816d4-31d8-477a-94d1-346368cfd65e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:11.306213 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e8e59-c1e9-4313-8009-94c405c01e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:11.307002 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e8e59-c1e9-4313-8009-94c405c01e8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:11.307313 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e8e59-c1e9-4313-8009-94c405c01e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:11.307591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e8e59-c1e9-4313-8009-94c405c01e8d admin admin] PATH_INFO: `/v3/projects/f2df1492593c40429906ed174145edb1` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:11.309469 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e8e59-c1e9-4313-8009-94c405c01e8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2df1492593c40429906ed174145edb1)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:11.311480 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e8e59-c1e9-4313-8009-94c405c01e8d admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:11.313911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:31:11.365620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-583e8e59-c1e9-4313-8009-94c405c01e8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2df1492593c40429906ed174145edb1', 'request_id': 'req-583e8e59-c1e9-4313-8009-94c405c01e8d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:11.372360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-583e8e59-c1e9-4313-8009-94c405c01e8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2df1492593c40429906ed174145edb1', 'request_id': 'req-583e8e59-c1e9-4313-8009-94c405c01e8d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:11.381020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-583e8e59-c1e9-4313-8009-94c405c01e8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2df1492593c40429906ed174145edb1', 'request_id': 'req-583e8e59-c1e9-4313-8009-94c405c01e8d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:11.392498 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-583e8e59-c1e9-4313-8009-94c405c01e8d admin admin] The token cache is being invalidate because project f2df1492593c40429906ed174145edb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:31:11.392783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-583e8e59-c1e9-4313-8009-94c405c01e8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:11.393429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-583e8e59-c1e9-4313-8009-94c405c01e8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:11.394898 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1258/2497] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:31:11 2022] DELETE /identity/v3/projects/f2df1492593c40429906ed174145edb1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:31:17.543353 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b7bc7-8db0-4534-8b93-b027f4bf0cba None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:17.543856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b7bc7-8db0-4534-8b93-b027f4bf0cba None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:17.543971 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b7bc7-8db0-4534-8b93-b027f4bf0cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:17.573371 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-d77b7bc7-8db0-4534-8b93-b027f4bf0cba None None] MFA Rules not processed for user `bc69cf267bea41bf890bc7551d84ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:31:17.575318 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d77b7bc7-8db0-4534-8b93-b027f4bf0cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:17.634399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1240/2498] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:31:17.679613 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e8e59-c1e9-4313-8009-94c405c01e8d admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:17.681007 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-583e8e59-c1e9-4313-8009-94c405c01e8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:17.748924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e8e59-c1e9-4313-8009-94c405c01e8d admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:17.753787 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae0626-be3a-4e64-8308-bd740ef59949 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975a8d7349994520bcbfb5f75e52aac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:17.754421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae0626-be3a-4e64-8308-bd740ef59949 service swift] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:17.754700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae0626-be3a-4e64-8308-bd740ef59949 service swift] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:17.754970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae0626-be3a-4e64-8308-bd740ef59949 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:17.763252 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae0626-be3a-4e64-8308-bd740ef59949 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:17.764797 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae0626-be3a-4e64-8308-bd740ef59949 service swift] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:17.772624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1259/2499] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 12:31:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 12:31:22.928967 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b7bc7-8db0-4534-8b93-b027f4bf0cba None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:22.930665 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d77b7bc7-8db0-4534-8b93-b027f4bf0cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:22.983907 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:31:22.996269 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b7bc7-8db0-4534-8b93-b027f4bf0cba None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:23.000507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8613f5-c432-4f83-b0b7-24367a7279fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:23.001117 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8613f5-c432-4f83-b0b7-24367a7279fd service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:23.001420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8613f5-c432-4f83-b0b7-24367a7279fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:23.001691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8613f5-c432-4f83-b0b7-24367a7279fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:23.003388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-3d8613f5-c432-4f83-b0b7-24367a7279fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:23.071073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8613f5-c432-4f83-b0b7-24367a7279fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:23.072610 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8613f5-c432-4f83-b0b7-24367a7279fd service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:23.083364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1241/2500] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:31:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:31:26.903110 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae0626-be3a-4e64-8308-bd740ef59949 service swift] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:26.905472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a8ae0626-be3a-4e64-8308-bd740ef59949 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:26.923742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae0626-be3a-4e64-8308-bd740ef59949 service swift] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:26.928577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6dc43-3c12-4f0f-985f-40d2dd907f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:26.929338 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6dc43-3c12-4f0f-985f-40d2dd907f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:26.929684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6dc43-3c12-4f0f-985f-40d2dd907f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:26.930028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6dc43-3c12-4f0f-985f-40d2dd907f0e admin admin] PATH_INFO: `/v3/users/cc96808ff3fb41f085f09aa943c3d016` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:26.931212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6dc43-3c12-4f0f-985f-40d2dd907f0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc96808ff3fb41f085f09aa943c3d016)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:26.933828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6dc43-3c12-4f0f-985f-40d2dd907f0e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:26.990861 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f6e6dc43-3c12-4f0f-985f-40d2dd907f0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc96808ff3fb41f085f09aa943c3d016'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:26.990861 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f6e6dc43-3c12-4f0f-985f-40d2dd907f0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc96808ff3fb41f085f09aa943c3d016'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:26.999797 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f6e6dc43-3c12-4f0f-985f-40d2dd907f0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc96808ff3fb41f085f09aa943c3d016'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:27.000513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f6e6dc43-3c12-4f0f-985f-40d2dd907f0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc96808ff3fb41f085f09aa943c3d016'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:27.010770 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f6e6dc43-3c12-4f0f-985f-40d2dd907f0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc96808ff3fb41f085f09aa943c3d016'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:27.024189 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:31:27.039009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1260/2501] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:31:26 2022] DELETE /identity/v3/users/cc96808ff3fb41f085f09aa943c3d016 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:31:27.277287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8613f5-c432-4f83-b0b7-24367a7279fd service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:27.290079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8613f5-c432-4f83-b0b7-24367a7279fd service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:27.371196 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a18b062-b571-484d-bdf9-d6b6c3adc12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:27.372008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18b062-b571-484d-bdf9-d6b6c3adc12f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:27.372394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18b062-b571-484d-bdf9-d6b6c3adc12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:27.372815 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a18b062-b571-484d-bdf9-d6b6c3adc12f admin admin] PATH_INFO: `/v3/projects/1318535cbbd6427c8c770bd5b256ac3e` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:27.374178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18b062-b571-484d-bdf9-d6b6c3adc12f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1318535cbbd6427c8c770bd5b256ac3e)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:27.377194 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a18b062-b571-484d-bdf9-d6b6c3adc12f admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:27.452515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-3a18b062-b571-484d-bdf9-d6b6c3adc12f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1318535cbbd6427c8c770bd5b256ac3e', 'request_id': 'req-3a18b062-b571-484d-bdf9-d6b6c3adc12f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:27.460980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-3a18b062-b571-484d-bdf9-d6b6c3adc12f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1318535cbbd6427c8c770bd5b256ac3e', 'request_id': 'req-3a18b062-b571-484d-bdf9-d6b6c3adc12f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:27.470898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-3a18b062-b571-484d-bdf9-d6b6c3adc12f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1318535cbbd6427c8c770bd5b256ac3e', 'request_id': 'req-3a18b062-b571-484d-bdf9-d6b6c3adc12f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:27.483266 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-3a18b062-b571-484d-bdf9-d6b6c3adc12f admin admin] The token cache is being invalidate because project 1318535cbbd6427c8c770bd5b256ac3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:31:27.483561 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-3a18b062-b571-484d-bdf9-d6b6c3adc12f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:27.484232 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-3a18b062-b571-484d-bdf9-d6b6c3adc12f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:31:27.485820 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1242/2502] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:31:27 2022] DELETE /identity/v3/projects/1318535cbbd6427c8c770bd5b256ac3e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:31:27.492027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8af5d-2a81-40fe-b5a2-fee979b12929 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:27.492310 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8af5d-2a81-40fe-b5a2-fee979b12929 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:27.492583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8af5d-2a81-40fe-b5a2-fee979b12929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:27.564228 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-89f8af5d-2a81-40fe-b5a2-fee979b12929 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:31:27.565549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-89f8af5d-2a81-40fe-b5a2-fee979b12929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:27.631576 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1261/2503] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:31:27.649694 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11ba93-f137-4791-8c16-7db4ddf9793d None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:27.649983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11ba93-f137-4791-8c16-7db4ddf9793d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:27.650267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11ba93-f137-4791-8c16-7db4ddf9793d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:27.674051 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-5b11ba93-f137-4791-8c16-7db4ddf9793d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:31:27.675374 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5b11ba93-f137-4791-8c16-7db4ddf9793d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:27.684702 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1243/2504] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:31:27.689147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8af5d-2a81-40fe-b5a2-fee979b12929 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:27.699480 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8af5d-2a81-40fe-b5a2-fee979b12929 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:27.704054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b301834-f62d-480b-9622-8a112c9db19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:27.704598 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b301834-f62d-480b-9622-8a112c9db19e admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:27.704916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b301834-f62d-480b-9622-8a112c9db19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:27.705204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b301834-f62d-480b-9622-8a112c9db19e admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:27.705848 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b301834-f62d-480b-9622-8a112c9db19e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:31:27.706149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b301834-f62d-480b-9622-8a112c9db19e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:27.707732 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b301834-f62d-480b-9622-8a112c9db19e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:27.726394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1262/2505] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:31:27.727508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11ba93-f137-4791-8c16-7db4ddf9793d None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:27.738633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11ba93-f137-4791-8c16-7db4ddf9793d None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:27.742519 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae54d24-80d4-406e-a19b-aa1a4e0cda6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:27.743136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae54d24-80d4-406e-a19b-aa1a4e0cda6e admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:27.743421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae54d24-80d4-406e-a19b-aa1a4e0cda6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:27.743688 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae54d24-80d4-406e-a19b-aa1a4e0cda6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:27.744462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae54d24-80d4-406e-a19b-aa1a4e0cda6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:27.746966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae54d24-80d4-406e-a19b-aa1a4e0cda6e admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:27.764002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:31:27.785064 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1244/2506] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:31:27 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:31:27.789120 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b301834-f62d-480b-9622-8a112c9db19e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:27.804590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b301834-f62d-480b-9622-8a112c9db19e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:27.872504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf4df1-aa1e-4907-b268-d2490bef2201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:27.873265 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf4df1-aa1e-4907-b268-d2490bef2201 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:27.873584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf4df1-aa1e-4907-b268-d2490bef2201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:27.873887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf4df1-aa1e-4907-b268-d2490bef2201 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:27.874537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf4df1-aa1e-4907-b268-d2490bef2201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:27.876508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf4df1-aa1e-4907-b268-d2490bef2201 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:27.906843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1263/2507] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:31:27 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:31:27.914548 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae54d24-80d4-406e-a19b-aa1a4e0cda6e admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:27.928264 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae54d24-80d4-406e-a19b-aa1a4e0cda6e admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:27.934936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62afcab-d5b4-4e74-9b09-fb8a36bef821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:27.936460 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62afcab-d5b4-4e74-9b09-fb8a36bef821 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:27.936742 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62afcab-d5b4-4e74-9b09-fb8a36bef821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:27.937016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62afcab-d5b4-4e74-9b09-fb8a36bef821 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:27.939919 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62afcab-d5b4-4e74-9b09-fb8a36bef821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:27.941198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62afcab-d5b4-4e74-9b09-fb8a36bef821 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:27.953611 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1245/2508] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:31:27.958078 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf4df1-aa1e-4907-b268-d2490bef2201 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:27.970715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf4df1-aa1e-4907-b268-d2490bef2201 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:27.975469 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697e6ba-f307-44cc-8279-d02df4151037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:27.976114 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697e6ba-f307-44cc-8279-d02df4151037 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:27.976400 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697e6ba-f307-44cc-8279-d02df4151037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:27.976677 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697e6ba-f307-44cc-8279-d02df4151037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:27.977440 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697e6ba-f307-44cc-8279-d02df4151037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:27.979132 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697e6ba-f307-44cc-8279-d02df4151037 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:27.991612 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1264/2509] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:31:27.997184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62afcab-d5b4-4e74-9b09-fb8a36bef821 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:28.007439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62afcab-d5b4-4e74-9b09-fb8a36bef821 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:28.014470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e010c-0636-43fb-afc8-069f72135363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:28.015989 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e010c-0636-43fb-afc8-069f72135363 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:28.016278 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e010c-0636-43fb-afc8-069f72135363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:28.016555 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e010c-0636-43fb-afc8-069f72135363 admin admin] PATH_INFO: `/v3/projects/77a0da6cfbd9415cbff856f9cfffee97/users/1885a2c4c42b47cb892b097557575642/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:28.033957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e010c-0636-43fb-afc8-069f72135363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77a0da6cfbd9415cbff856f9cfffee97, user_id=1885a2c4c42b47cb892b097557575642, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:28.036677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e010c-0636-43fb-afc8-069f72135363 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:28.054654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1246/2510] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:31:27 2022] PUT /identity/v3/projects/77a0da6cfbd9415cbff856f9cfffee97/users/1885a2c4c42b47cb892b097557575642/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:31:28.060488 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055d828-5984-414d-8690-ed9f4bf16941 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:28.060488 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055d828-5984-414d-8690-ed9f4bf16941 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:28.060743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055d828-5984-414d-8690-ed9f4bf16941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:28.082623 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-4055d828-5984-414d-8690-ed9f4bf16941 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:31:28.083855 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-4055d828-5984-414d-8690-ed9f4bf16941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:28.114278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:31:28.160675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1265/2511] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:31:28.173627 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e010c-0636-43fb-afc8-069f72135363 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:28.176384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-4b5e010c-0636-43fb-afc8-069f72135363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:28.248817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e010c-0636-43fb-afc8-069f72135363 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:28.252493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f768b0-17f5-48f6-a091-062661d2dc5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:28.253104 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f768b0-17f5-48f6-a091-062661d2dc5c service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:28.253415 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f768b0-17f5-48f6-a091-062661d2dc5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:28.253687 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f768b0-17f5-48f6-a091-062661d2dc5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:28.263964 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f768b0-17f5-48f6-a091-062661d2dc5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:28.265434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f768b0-17f5-48f6-a091-062661d2dc5c service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:28.276273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1247/2512] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:31:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:31:32.736073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fb7f3-d0a0-412c-b2e4-9b7f4630f83c None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:32.736575 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fb7f3-d0a0-412c-b2e4-9b7f4630f83c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:32.736662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fb7f3-d0a0-412c-b2e4-9b7f4630f83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:32.767016 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-aa6fb7f3-d0a0-412c-b2e4-9b7f4630f83c None None] MFA Rules not processed for user `1885a2c4c42b47cb892b097557575642`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:31:32.768210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-aa6fb7f3-d0a0-412c-b2e4-9b7f4630f83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:32.876065 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1266/2513] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:31:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:31:32.891984 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f768b0-17f5-48f6-a091-062661d2dc5c service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:32.893594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-98f768b0-17f5-48f6-a091-062661d2dc5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:33.011475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f768b0-17f5-48f6-a091-062661d2dc5c service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:33.019265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33d7c6-f4ef-4081-a1f4-6526d11fa497 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:33.020136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33d7c6-f4ef-4081-a1f4-6526d11fa497 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:33.020573 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33d7c6-f4ef-4081-a1f4-6526d11fa497 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:33.021001 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33d7c6-f4ef-4081-a1f4-6526d11fa497 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:33.031236 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33d7c6-f4ef-4081-a1f4-6526d11fa497 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:33.033629 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33d7c6-f4ef-4081-a1f4-6526d11fa497 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:33.043891 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1248/2514] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:31:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:31:34.624986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c8bda-3b23-4c08-ab81-e8c329923977 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:34.625527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c8bda-3b23-4c08-ab81-e8c329923977 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:34.625698 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c8bda-3b23-4c08-ab81-e8c329923977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:34.653642 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-755c8bda-3b23-4c08-ab81-e8c329923977 None None] MFA Rules not processed for user `1885a2c4c42b47cb892b097557575642`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:31:34.655523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-755c8bda-3b23-4c08-ab81-e8c329923977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:34.666464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1267/2515] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:31:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:31:36.690332 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d70e7a-9261-421e-bd23-7e6c5918fccf None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:36.690832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d70e7a-9261-421e-bd23-7e6c5918fccf None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:36.691592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d70e7a-9261-421e-bd23-7e6c5918fccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:36.744431 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-72d70e7a-9261-421e-bd23-7e6c5918fccf None None] MFA Rules not processed for user `bc69cf267bea41bf890bc7551d84ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:31:36.749527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-72d70e7a-9261-421e-bd23-7e6c5918fccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:36.811523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:31:36.864927 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1249/2516] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 12:31:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:31:36.878990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c8bda-3b23-4c08-ab81-e8c329923977 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:36.880754 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-755c8bda-3b23-4c08-ab81-e8c329923977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:36.968318 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c8bda-3b23-4c08-ab81-e8c329923977 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:36.974089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bfa7d-dc00-49a1-823e-d0856ebc6ca3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975a8d7349994520bcbfb5f75e52aac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:36.974718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bfa7d-dc00-49a1-823e-d0856ebc6ca3 service swift] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:36.975003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bfa7d-dc00-49a1-823e-d0856ebc6ca3 service swift] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:36.975277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bfa7d-dc00-49a1-823e-d0856ebc6ca3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:36.985298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485bfa7d-dc00-49a1-823e-d0856ebc6ca3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:36.986984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485bfa7d-dc00-49a1-823e-d0856ebc6ca3 service swift] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:36.997076 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1268/2517] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 12:31:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 12:31:46.865553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=12 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:31:46.992875 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=81 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:31:57.604573 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1fa9e-2d49-4ec2-83f5-ec642a20c455 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:57.605074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1fa9e-2d49-4ec2-83f5-ec642a20c455 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:57.605211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1fa9e-2d49-4ec2-83f5-ec642a20c455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:57.624744 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-7cb1fa9e-2d49-4ec2-83f5-ec642a20c455 None None] MFA Rules not processed for user `bc69cf267bea41bf890bc7551d84ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:31:57.625962 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7cb1fa9e-2d49-4ec2-83f5-ec642a20c455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:31:57.633436 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1250/2518] 10.222.0.60 () {58 vars in 967 bytes} [Wed Sep 28 12:31:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:31:57.647581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bfa7d-dc00-49a1-823e-d0856ebc6ca3 service swift] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:31:57.658163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bfa7d-dc00-49a1-823e-d0856ebc6ca3 service swift] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:31:57.663544 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10871189-6b70-4dd2-9f28-77482cfc0c2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975a8d7349994520bcbfb5f75e52aac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:31:57.664146 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10871189-6b70-4dd2-9f28-77482cfc0c2f service swift] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:31:57.664423 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10871189-6b70-4dd2-9f28-77482cfc0c2f service swift] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:31:57.664689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10871189-6b70-4dd2-9f28-77482cfc0c2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:31:57.673436 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10871189-6b70-4dd2-9f28-77482cfc0c2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:31:57.674990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10871189-6b70-4dd2-9f28-77482cfc0c2f service swift] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:31:57.682766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1269/2519] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 12:31:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 12:32:07.618845 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:32:07.679592 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=6 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:32:34.965523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1fa9e-2d49-4ec2-83f5-ec642a20c455 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:34.977344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1fa9e-2d49-4ec2-83f5-ec642a20c455 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:34.980902 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb68e27-af7d-43c0-8095-b48a3d23165f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:34.981538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb68e27-af7d-43c0-8095-b48a3d23165f service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:34.981817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb68e27-af7d-43c0-8095-b48a3d23165f service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:34.982112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb68e27-af7d-43c0-8095-b48a3d23165f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:34.983834 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-3eb68e27-af7d-43c0-8095-b48a3d23165f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:34.996346 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb68e27-af7d-43c0-8095-b48a3d23165f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:34.997782 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb68e27-af7d-43c0-8095-b48a3d23165f service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:35.007517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1251/2520] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:32:38.046089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10871189-6b70-4dd2-9f28-77482cfc0c2f service swift] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:38.047584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-10871189-6b70-4dd2-9f28-77482cfc0c2f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:38.120600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb68e27-af7d-43c0-8095-b48a3d23165f service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:38.122024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-3eb68e27-af7d-43c0-8095-b48a3d23165f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:38.129671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10871189-6b70-4dd2-9f28-77482cfc0c2f service swift] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:38.135487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d51402-8ade-422b-a7c1-0407f6850ba8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:38.136129 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d51402-8ade-422b-a7c1-0407f6850ba8 service placement] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:38.136301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb68e27-af7d-43c0-8095-b48a3d23165f service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:38.136637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d51402-8ade-422b-a7c1-0407f6850ba8 service placement] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:38.136915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d51402-8ade-422b-a7c1-0407f6850ba8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:38.139492 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-09d51402-8ade-422b-a7c1-0407f6850ba8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:38.141098 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb3412-1452-4daa-a223-ebd2724ff040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:38.141806 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb3412-1452-4daa-a223-ebd2724ff040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:38.142089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb3412-1452-4daa-a223-ebd2724ff040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:38.142540 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb3412-1452-4daa-a223-ebd2724ff040 admin admin] PATH_INFO: `/v3/users/8429a502bca744fc9f5d5701fe9c766c` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:38.153522 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d51402-8ade-422b-a7c1-0407f6850ba8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:38.155242 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d51402-8ade-422b-a7c1-0407f6850ba8 service placement] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:38.165223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb3412-1452-4daa-a223-ebd2724ff040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8429a502bca744fc9f5d5701fe9c766c)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:38.170072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb3412-1452-4daa-a223-ebd2724ff040 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:38.170789 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1270/2521] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:32:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:32:38.233047 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-3cdb3412-1452-4daa-a223-ebd2724ff040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8429a502bca744fc9f5d5701fe9c766c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:38.234061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-3cdb3412-1452-4daa-a223-ebd2724ff040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8429a502bca744fc9f5d5701fe9c766c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:38.248015 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-3cdb3412-1452-4daa-a223-ebd2724ff040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8429a502bca744fc9f5d5701fe9c766c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:38.256996 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-3cdb3412-1452-4daa-a223-ebd2724ff040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8429a502bca744fc9f5d5701fe9c766c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:38.257654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-3cdb3412-1452-4daa-a223-ebd2724ff040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8429a502bca744fc9f5d5701fe9c766c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:38.271813 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1252/2522] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:32:38 2022] DELETE /identity/v3/users/8429a502bca744fc9f5d5701fe9c766c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:32:38.450181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d51402-8ade-422b-a7c1-0407f6850ba8 service placement] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:38.462747 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d51402-8ade-422b-a7c1-0407f6850ba8 service placement] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:38.523813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389db78-4879-4e21-8548-f1efbf249abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:38.524392 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389db78-4879-4e21-8548-f1efbf249abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:38.524675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389db78-4879-4e21-8548-f1efbf249abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:38.524951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389db78-4879-4e21-8548-f1efbf249abb admin admin] PATH_INFO: `/v3/projects/8f9ceb72510849c9a1cce4cde67d06d2` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:38.535754 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389db78-4879-4e21-8548-f1efbf249abb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f9ceb72510849c9a1cce4cde67d06d2)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:38.537647 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389db78-4879-4e21-8548-f1efbf249abb admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:38.584108 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2389db78-4879-4e21-8548-f1efbf249abb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f9ceb72510849c9a1cce4cde67d06d2', 'request_id': 'req-2389db78-4879-4e21-8548-f1efbf249abb'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:38.590671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2389db78-4879-4e21-8548-f1efbf249abb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f9ceb72510849c9a1cce4cde67d06d2', 'request_id': 'req-2389db78-4879-4e21-8548-f1efbf249abb'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:38.598384 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2389db78-4879-4e21-8548-f1efbf249abb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f9ceb72510849c9a1cce4cde67d06d2', 'request_id': 'req-2389db78-4879-4e21-8548-f1efbf249abb'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:38.608307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2389db78-4879-4e21-8548-f1efbf249abb admin admin] The token cache is being invalidate because project 8f9ceb72510849c9a1cce4cde67d06d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:32:38.608592 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2389db78-4879-4e21-8548-f1efbf249abb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:38.609244 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2389db78-4879-4e21-8548-f1efbf249abb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:38.610711 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1271/2523] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:32:38 2022] DELETE /identity/v3/projects/8f9ceb72510849c9a1cce4cde67d06d2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:32:38.616329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76a7d5-8f47-4f7f-a035-9101611f70c9 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:38.616612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76a7d5-8f47-4f7f-a035-9101611f70c9 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:38.616892 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76a7d5-8f47-4f7f-a035-9101611f70c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:38.636757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-7b76a7d5-8f47-4f7f-a035-9101611f70c9 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:32:38.637984 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7b76a7d5-8f47-4f7f-a035-9101611f70c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:38.696810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1253/2524] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:32:38.704818 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba26d50-71bf-4572-970d-23aaa0d3eddf None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:38.705121 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba26d50-71bf-4572-970d-23aaa0d3eddf None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:38.705429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba26d50-71bf-4572-970d-23aaa0d3eddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:38.725229 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-6ba26d50-71bf-4572-970d-23aaa0d3eddf None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:32:38.726411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-6ba26d50-71bf-4572-970d-23aaa0d3eddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:38.734108 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1272/2525] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:32:38.738876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76a7d5-8f47-4f7f-a035-9101611f70c9 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:38.748396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76a7d5-8f47-4f7f-a035-9101611f70c9 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:38.753002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbbdad-fb94-4e51-9b23-dec4cc799fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:38.753626 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbbdad-fb94-4e51-9b23-dec4cc799fbb admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:38.753909 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbbdad-fb94-4e51-9b23-dec4cc799fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:38.754206 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbbdad-fb94-4e51-9b23-dec4cc799fbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:38.754842 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdbbdad-fb94-4e51-9b23-dec4cc799fbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:32:38.755146 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdbbdad-fb94-4e51-9b23-dec4cc799fbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:38.756763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdbbdad-fb94-4e51-9b23-dec4cc799fbb admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:38.768109 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1254/2526] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:32:38.772265 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba26d50-71bf-4572-970d-23aaa0d3eddf None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:38.777838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:32:38.781534 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba26d50-71bf-4572-970d-23aaa0d3eddf None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:38.787966 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5b8ec-2fed-4267-9af3-16fcfd62e702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:38.788794 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5b8ec-2fed-4267-9af3-16fcfd62e702 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:38.789204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5b8ec-2fed-4267-9af3-16fcfd62e702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:38.789624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b5b8ec-2fed-4267-9af3-16fcfd62e702 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:38.790470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5b8ec-2fed-4267-9af3-16fcfd62e702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:38.792383 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b5b8ec-2fed-4267-9af3-16fcfd62e702 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:38.828695 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1273/2527] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:32:38 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:32:38.832824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbbdad-fb94-4e51-9b23-dec4cc799fbb admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:38.843125 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbbdad-fb94-4e51-9b23-dec4cc799fbb admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:38.875470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:32:38.902376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62501c34-e963-4eaa-8c3f-b19e717557ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:38.902972 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62501c34-e963-4eaa-8c3f-b19e717557ad admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:38.903248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62501c34-e963-4eaa-8c3f-b19e717557ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:38.903516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62501c34-e963-4eaa-8c3f-b19e717557ad admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:38.904133 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62501c34-e963-4eaa-8c3f-b19e717557ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:38.906073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62501c34-e963-4eaa-8c3f-b19e717557ad admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:38.926917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1255/2528] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:32:38 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:32:38.931204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5b8ec-2fed-4267-9af3-16fcfd62e702 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:38.942218 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b5b8ec-2fed-4267-9af3-16fcfd62e702 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:38.947259 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18f8ac-7e11-445f-b1fe-e7aaedf14d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:38.947971 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18f8ac-7e11-445f-b1fe-e7aaedf14d2a admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:38.948290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18f8ac-7e11-445f-b1fe-e7aaedf14d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:38.948629 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18f8ac-7e11-445f-b1fe-e7aaedf14d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:38.949471 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18f8ac-7e11-445f-b1fe-e7aaedf14d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:38.951233 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18f8ac-7e11-445f-b1fe-e7aaedf14d2a admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:38.962242 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1274/2529] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:32:38.966277 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62501c34-e963-4eaa-8c3f-b19e717557ad admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:38.976435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62501c34-e963-4eaa-8c3f-b19e717557ad admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:38.981019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea8846-ae20-41f6-8e85-c104e1d7f044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:38.981676 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea8846-ae20-41f6-8e85-c104e1d7f044 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:38.981955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea8846-ae20-41f6-8e85-c104e1d7f044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:38.982228 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea8846-ae20-41f6-8e85-c104e1d7f044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:38.982861 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea8846-ae20-41f6-8e85-c104e1d7f044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:38.984421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea8846-ae20-41f6-8e85-c104e1d7f044 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:38.994156 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1256/2530] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:32:38.998050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18f8ac-7e11-445f-b1fe-e7aaedf14d2a admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:39.008384 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18f8ac-7e11-445f-b1fe-e7aaedf14d2a admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:39.012947 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478bb41-50ad-449a-a0c7-bce843e8ae27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:39.013643 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478bb41-50ad-449a-a0c7-bce843e8ae27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:39.013909 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478bb41-50ad-449a-a0c7-bce843e8ae27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:39.014186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478bb41-50ad-449a-a0c7-bce843e8ae27 admin admin] PATH_INFO: `/v3/projects/be2bf35f65b44ba1beb6e2815ac40c33/users/cd7d7fd987784c83b773d5e61f506ab3/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:39.029398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478bb41-50ad-449a-a0c7-bce843e8ae27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be2bf35f65b44ba1beb6e2815ac40c33, user_id=cd7d7fd987784c83b773d5e61f506ab3, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:39.031962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478bb41-50ad-449a-a0c7-bce843e8ae27 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:39.045538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1275/2531] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:32:38 2022] PUT /identity/v3/projects/be2bf35f65b44ba1beb6e2815ac40c33/users/cd7d7fd987784c83b773d5e61f506ab3/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:32:39.057478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670e36e-f5ce-4908-b976-46126cead990 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:39.057823 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670e36e-f5ce-4908-b976-46126cead990 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:39.058105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670e36e-f5ce-4908-b976-46126cead990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:39.087113 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-4670e36e-f5ce-4908-b976-46126cead990 None None] MFA Rules not processed for user `cd7d7fd987784c83b773d5e61f506ab3`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:32:39.088324 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-4670e36e-f5ce-4908-b976-46126cead990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:39.151545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1257/2532] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:32:39.156082 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478bb41-50ad-449a-a0c7-bce843e8ae27 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:39.216830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478bb41-50ad-449a-a0c7-bce843e8ae27 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:39.222093 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f9e7f-163b-4784-9d41-7f8fc8f416ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:39.222694 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f9e7f-163b-4784-9d41-7f8fc8f416ba admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:39.222974 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f9e7f-163b-4784-9d41-7f8fc8f416ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:39.223249 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f9e7f-163b-4784-9d41-7f8fc8f416ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:39.223886 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f9e7f-163b-4784-9d41-7f8fc8f416ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:39.225809 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f9e7f-163b-4784-9d41-7f8fc8f416ba admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:39.253736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1276/2533] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:32:39 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:32:39.257791 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670e36e-f5ce-4908-b976-46126cead990 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:39.270407 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670e36e-f5ce-4908-b976-46126cead990 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:39.327634 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7214e9-8c92-40f5-856a-bc409e0b7236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:39.328159 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7214e9-8c92-40f5-856a-bc409e0b7236 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:39.328441 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7214e9-8c92-40f5-856a-bc409e0b7236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:39.328716 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7214e9-8c92-40f5-856a-bc409e0b7236 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:39.329380 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7214e9-8c92-40f5-856a-bc409e0b7236 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:39.331579 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7214e9-8c92-40f5-856a-bc409e0b7236 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:39.352402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1258/2534] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:32:39 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:32:39.356444 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f9e7f-163b-4784-9d41-7f8fc8f416ba admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:39.367412 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f9e7f-163b-4784-9d41-7f8fc8f416ba admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:39.372022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e55ef-7110-48cd-9db3-59af9110c8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:39.372620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e55ef-7110-48cd-9db3-59af9110c8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:39.372899 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e55ef-7110-48cd-9db3-59af9110c8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:39.373201 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e55ef-7110-48cd-9db3-59af9110c8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:39.373833 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e55ef-7110-48cd-9db3-59af9110c8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:39.375409 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e55ef-7110-48cd-9db3-59af9110c8e7 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:39.386526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1277/2535] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:32:39.390510 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7214e9-8c92-40f5-856a-bc409e0b7236 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:39.400310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7214e9-8c92-40f5-856a-bc409e0b7236 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:39.404910 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafc34b-d7c6-403c-87f5-a2b1d57aaca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:39.405590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafc34b-d7c6-403c-87f5-a2b1d57aaca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:39.405868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafc34b-d7c6-403c-87f5-a2b1d57aaca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:39.406136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafc34b-d7c6-403c-87f5-a2b1d57aaca1 admin admin] PATH_INFO: `/v3/projects/a6055d70010f45fc9267801519c1553d/users/f5aa047abd5941319c9d317d9f2b2561/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:39.421068 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafc34b-d7c6-403c-87f5-a2b1d57aaca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6055d70010f45fc9267801519c1553d, user_id=f5aa047abd5941319c9d317d9f2b2561, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:39.423697 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafc34b-d7c6-403c-87f5-a2b1d57aaca1 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:39.437388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1259/2536] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:32:39 2022] PUT /identity/v3/projects/a6055d70010f45fc9267801519c1553d/users/f5aa047abd5941319c9d317d9f2b2561/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:32:39.441586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e55ef-7110-48cd-9db3-59af9110c8e7 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:39.451546 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e55ef-7110-48cd-9db3-59af9110c8e7 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:39.507011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50a6dc-3f87-40f5-9fbf-8ef40ce392cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:39.507613 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a6dc-3f87-40f5-9fbf-8ef40ce392cc admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:39.507890 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a6dc-3f87-40f5-9fbf-8ef40ce392cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:39.508158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a6dc-3f87-40f5-9fbf-8ef40ce392cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:39.508780 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50a6dc-3f87-40f5-9fbf-8ef40ce392cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:39.510454 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50a6dc-3f87-40f5-9fbf-8ef40ce392cc admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:39.520033 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1278/2537] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:32:39.524149 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafc34b-d7c6-403c-87f5-a2b1d57aaca1 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:39.534774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafc34b-d7c6-403c-87f5-a2b1d57aaca1 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:39.539551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63739b4-206a-4392-92ff-3b9d2c41974c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:39.540210 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63739b4-206a-4392-92ff-3b9d2c41974c admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:39.540494 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63739b4-206a-4392-92ff-3b9d2c41974c admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:39.540771 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63739b4-206a-4392-92ff-3b9d2c41974c admin admin] PATH_INFO: `/v3/projects/a6055d70010f45fc9267801519c1553d/users/f5aa047abd5941319c9d317d9f2b2561/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:39.542762 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63739b4-206a-4392-92ff-3b9d2c41974c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6055d70010f45fc9267801519c1553d, user_id=f5aa047abd5941319c9d317d9f2b2561, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:39.545456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63739b4-206a-4392-92ff-3b9d2c41974c admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:39.548527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:32:39.557825 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1260/2538] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:32:39 2022] PUT /identity/v3/projects/a6055d70010f45fc9267801519c1553d/users/f5aa047abd5941319c9d317d9f2b2561/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:32:39.568008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f5c7c-d886-4b04-8d1b-05e55f3d9807 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:39.568337 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f5c7c-d886-4b04-8d1b-05e55f3d9807 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:39.568570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f5c7c-d886-4b04-8d1b-05e55f3d9807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:39.589774 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:32:39.599895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-1a6f5c7c-d886-4b04-8d1b-05e55f3d9807 None None] MFA Rules not processed for user `f5aa047abd5941319c9d317d9f2b2561`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:32:39.601121 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-1a6f5c7c-d886-4b04-8d1b-05e55f3d9807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:39.667229 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1279/2539] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:32:39.679435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63739b4-206a-4392-92ff-3b9d2c41974c admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:39.681200 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f63739b4-206a-4392-92ff-3b9d2c41974c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:39.759547 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63739b4-206a-4392-92ff-3b9d2c41974c admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:39.763284 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32702413-d49a-4327-92a5-90d5c1d2ad32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:39.763882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32702413-d49a-4327-92a5-90d5c1d2ad32 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:39.764167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32702413-d49a-4327-92a5-90d5c1d2ad32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:39.764434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32702413-d49a-4327-92a5-90d5c1d2ad32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:39.773730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32702413-d49a-4327-92a5-90d5c1d2ad32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:39.775153 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32702413-d49a-4327-92a5-90d5c1d2ad32 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:39.785453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1261/2540] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:32:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:32:39.829671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f5c7c-d886-4b04-8d1b-05e55f3d9807 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:39.831678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-1a6f5c7c-d886-4b04-8d1b-05e55f3d9807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:39.843958 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f5c7c-d886-4b04-8d1b-05e55f3d9807 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:39.848499 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa9570-6af9-4899-83c4-0d104fa7b2b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:39.849126 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa9570-6af9-4899-83c4-0d104fa7b2b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:39.849437 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa9570-6af9-4899-83c4-0d104fa7b2b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:39.849708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa9570-6af9-4899-83c4-0d104fa7b2b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:39.904235 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa9570-6af9-4899-83c4-0d104fa7b2b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:39.905677 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa9570-6af9-4899-83c4-0d104fa7b2b2 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:39.915116 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1280/2541] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:32:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:32:43.208498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32702413-d49a-4327-92a5-90d5c1d2ad32 service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:43.270311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32702413-d49a-4327-92a5-90d5c1d2ad32 service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:43.274916 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147443b-dc07-49c6-934a-9ef851f5ecb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:43.275564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147443b-dc07-49c6-934a-9ef851f5ecb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:43.275825 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147443b-dc07-49c6-934a-9ef851f5ecb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:43.276101 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147443b-dc07-49c6-934a-9ef851f5ecb7 admin admin] PATH_INFO: `/v3/users/cd7d7fd987784c83b773d5e61f506ab3` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:43.277264 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147443b-dc07-49c6-934a-9ef851f5ecb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd7d7fd987784c83b773d5e61f506ab3)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:43.279175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147443b-dc07-49c6-934a-9ef851f5ecb7 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:43.326788 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-9147443b-dc07-49c6-934a-9ef851f5ecb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd7d7fd987784c83b773d5e61f506ab3'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:43.327432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-9147443b-dc07-49c6-934a-9ef851f5ecb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd7d7fd987784c83b773d5e61f506ab3'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:43.339155 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-9147443b-dc07-49c6-934a-9ef851f5ecb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd7d7fd987784c83b773d5e61f506ab3'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:43.347377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-9147443b-dc07-49c6-934a-9ef851f5ecb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd7d7fd987784c83b773d5e61f506ab3'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:43.348024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-9147443b-dc07-49c6-934a-9ef851f5ecb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd7d7fd987784c83b773d5e61f506ab3'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:43.355895 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:32:43.361000 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1262/2542] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:32:43 2022] DELETE /identity/v3/users/cd7d7fd987784c83b773d5e61f506ab3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:32:43.366615 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cffe7c2-99f5-445a-9d6c-f125b46633e5 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:43.367089 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cffe7c2-99f5-445a-9d6c-f125b46633e5 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:43.367217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cffe7c2-99f5-445a-9d6c-f125b46633e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:43.386313 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-1cffe7c2-99f5-445a-9d6c-f125b46633e5 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:32:43.387567 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-1cffe7c2-99f5-445a-9d6c-f125b46633e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:43.444211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1281/2543] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:32:43.453750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147443b-dc07-49c6-934a-9ef851f5ecb7 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:43.455175 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-9147443b-dc07-49c6-934a-9ef851f5ecb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:43.511868 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147443b-dc07-49c6-934a-9ef851f5ecb7 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:43.515533 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e68f1-cd67-451c-8794-aa45bc04b092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:43.516136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e68f1-cd67-451c-8794-aa45bc04b092 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:43.516415 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e68f1-cd67-451c-8794-aa45bc04b092 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:43.516688 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e68f1-cd67-451c-8794-aa45bc04b092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:43.525657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e68f1-cd67-451c-8794-aa45bc04b092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:43.527085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e68f1-cd67-451c-8794-aa45bc04b092 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:43.537382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1263/2544] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:32:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:32:43.805330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cffe7c2-99f5-445a-9d6c-f125b46633e5 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:43.806814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-1cffe7c2-99f5-445a-9d6c-f125b46633e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:43.821829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cffe7c2-99f5-445a-9d6c-f125b46633e5 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:43.826930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:43.827740 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:43.828157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:43.828291 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca admin admin] PATH_INFO: `/v3/projects/be2bf35f65b44ba1beb6e2815ac40c33` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:43.829450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=be2bf35f65b44ba1beb6e2815ac40c33)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:43.831727 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:43.867913 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:32:43.883667 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be2bf35f65b44ba1beb6e2815ac40c33', 'request_id': 'req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:43.889708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be2bf35f65b44ba1beb6e2815ac40c33', 'request_id': 'req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:43.896804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be2bf35f65b44ba1beb6e2815ac40c33', 'request_id': 'req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:43.907001 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca admin admin] The token cache is being invalidate because project be2bf35f65b44ba1beb6e2815ac40c33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:32:43.907294 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:43.907902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:43.909920 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1282/2545] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:32:43 2022] DELETE /identity/v3/projects/be2bf35f65b44ba1beb6e2815ac40c33 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:32:43.913882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e68f1-cd67-451c-8794-aa45bc04b092 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:43.923409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e68f1-cd67-451c-8794-aa45bc04b092 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:43.979480 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b155058-44ba-4bbb-9367-0e4b5fa00817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:43.980029 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b155058-44ba-4bbb-9367-0e4b5fa00817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:43.980308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b155058-44ba-4bbb-9367-0e4b5fa00817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:43.980600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b155058-44ba-4bbb-9367-0e4b5fa00817 admin admin] PATH_INFO: `/v3/users/f5aa047abd5941319c9d317d9f2b2561` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:43.981720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b155058-44ba-4bbb-9367-0e4b5fa00817 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5aa047abd5941319c9d317d9f2b2561)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:43.983625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b155058-44ba-4bbb-9367-0e4b5fa00817 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:44.027554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-3b155058-44ba-4bbb-9367-0e4b5fa00817 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5aa047abd5941319c9d317d9f2b2561'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:44.028192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-3b155058-44ba-4bbb-9367-0e4b5fa00817 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5aa047abd5941319c9d317d9f2b2561'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:44.040319 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-3b155058-44ba-4bbb-9367-0e4b5fa00817 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5aa047abd5941319c9d317d9f2b2561'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:44.046826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-3b155058-44ba-4bbb-9367-0e4b5fa00817 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5aa047abd5941319c9d317d9f2b2561'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:44.047434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-3b155058-44ba-4bbb-9367-0e4b5fa00817 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5aa047abd5941319c9d317d9f2b2561'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:44.060679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1264/2546] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:32:43 2022] DELETE /identity/v3/users/f5aa047abd5941319c9d317d9f2b2561 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:32:44.326642 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:44.328092 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:44.343713 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c05f2a-1b0d-42ef-8aa1-7d42352a01ca admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:44.421991 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cce18c-4f16-48ab-9c24-32bac04b63f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:44.422796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cce18c-4f16-48ab-9c24-32bac04b63f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:44.422928 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cce18c-4f16-48ab-9c24-32bac04b63f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:44.423158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cce18c-4f16-48ab-9c24-32bac04b63f7 admin admin] PATH_INFO: `/v3/projects/a6055d70010f45fc9267801519c1553d` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:44.424740 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cce18c-4f16-48ab-9c24-32bac04b63f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6055d70010f45fc9267801519c1553d)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:44.426676 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cce18c-4f16-48ab-9c24-32bac04b63f7 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:44.473323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d1cce18c-4f16-48ab-9c24-32bac04b63f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6055d70010f45fc9267801519c1553d', 'request_id': 'req-d1cce18c-4f16-48ab-9c24-32bac04b63f7'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:44.479347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d1cce18c-4f16-48ab-9c24-32bac04b63f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6055d70010f45fc9267801519c1553d', 'request_id': 'req-d1cce18c-4f16-48ab-9c24-32bac04b63f7'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:44.486873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d1cce18c-4f16-48ab-9c24-32bac04b63f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6055d70010f45fc9267801519c1553d', 'request_id': 'req-d1cce18c-4f16-48ab-9c24-32bac04b63f7'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:44.497036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d1cce18c-4f16-48ab-9c24-32bac04b63f7 admin admin] The token cache is being invalidate because project a6055d70010f45fc9267801519c1553d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:32:44.497361 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d1cce18c-4f16-48ab-9c24-32bac04b63f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:44.497990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d1cce18c-4f16-48ab-9c24-32bac04b63f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:44.499461 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1283/2547] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:32:44 2022] DELETE /identity/v3/projects/a6055d70010f45fc9267801519c1553d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:32:44.504782 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a44580-6ad8-4f05-bd57-4d8f29dca2e2 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:44.505295 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a44580-6ad8-4f05-bd57-4d8f29dca2e2 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:44.505429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a44580-6ad8-4f05-bd57-4d8f29dca2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:44.525123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-45a44580-6ad8-4f05-bd57-4d8f29dca2e2 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:32:44.526333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-45a44580-6ad8-4f05-bd57-4d8f29dca2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:44.539480 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:32:44.588083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1265/2548] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:32:44.596734 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec170ef4-aead-41e2-b787-c3494ae180fa None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:44.597021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec170ef4-aead-41e2-b787-c3494ae180fa None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:44.597329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec170ef4-aead-41e2-b787-c3494ae180fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:44.616664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-ec170ef4-aead-41e2-b787-c3494ae180fa None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:32:44.617897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-ec170ef4-aead-41e2-b787-c3494ae180fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:44.625721 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1284/2549] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:32:44.629857 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a44580-6ad8-4f05-bd57-4d8f29dca2e2 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:44.639340 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a44580-6ad8-4f05-bd57-4d8f29dca2e2 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:44.644529 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af39ae-f674-4d65-9e2a-8708c57afe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:44.645124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af39ae-f674-4d65-9e2a-8708c57afe5a admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:44.645436 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af39ae-f674-4d65-9e2a-8708c57afe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:44.645707 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af39ae-f674-4d65-9e2a-8708c57afe5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:44.646338 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af39ae-f674-4d65-9e2a-8708c57afe5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:32:44.646638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af39ae-f674-4d65-9e2a-8708c57afe5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:44.648238 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8af39ae-f674-4d65-9e2a-8708c57afe5a admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:44.660192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1266/2550] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:32:44.664508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec170ef4-aead-41e2-b787-c3494ae180fa None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:44.673978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec170ef4-aead-41e2-b787-c3494ae180fa None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:44.683142 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4469b7-6b7f-4d5b-b842-b6041c55a965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:44.683749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4469b7-6b7f-4d5b-b842-b6041c55a965 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:44.684031 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4469b7-6b7f-4d5b-b842-b6041c55a965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:44.684302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4469b7-6b7f-4d5b-b842-b6041c55a965 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:44.684939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4469b7-6b7f-4d5b-b842-b6041c55a965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:44.686861 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4469b7-6b7f-4d5b-b842-b6041c55a965 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:44.719403 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1285/2551] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:32:44 2022] POST /identity/v3/projects => generated 407 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:32:44.723744 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af39ae-f674-4d65-9e2a-8708c57afe5a admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:44.733717 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af39ae-f674-4d65-9e2a-8708c57afe5a admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:44.789784 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e602b32-9773-4c6b-baee-71ad4ef42d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:44.790312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e602b32-9773-4c6b-baee-71ad4ef42d35 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:44.790591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e602b32-9773-4c6b-baee-71ad4ef42d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:44.790861 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e602b32-9773-4c6b-baee-71ad4ef42d35 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:44.791493 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e602b32-9773-4c6b-baee-71ad4ef42d35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:44.793453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e602b32-9773-4c6b-baee-71ad4ef42d35 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:44.813864 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1267/2552] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:32:44 2022] POST /identity/v3/users => generated 304 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:32:44.818494 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4469b7-6b7f-4d5b-b842-b6041c55a965 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:44.828658 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4469b7-6b7f-4d5b-b842-b6041c55a965 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:44.834371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2405be6-4504-4753-98e2-1f2236f716a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:44.834957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2405be6-4504-4753-98e2-1f2236f716a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:44.835234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2405be6-4504-4753-98e2-1f2236f716a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:44.835501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2405be6-4504-4753-98e2-1f2236f716a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:44.836125 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2405be6-4504-4753-98e2-1f2236f716a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:44.837720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2405be6-4504-4753-98e2-1f2236f716a9 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:44.847919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1286/2553] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:32:44.851837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e602b32-9773-4c6b-baee-71ad4ef42d35 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:44.861863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e602b32-9773-4c6b-baee-71ad4ef42d35 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:44.866526 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182a25e-2cb8-455d-b9ea-aac0373580dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:44.867119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182a25e-2cb8-455d-b9ea-aac0373580dd admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:44.867399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182a25e-2cb8-455d-b9ea-aac0373580dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:44.867818 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182a25e-2cb8-455d-b9ea-aac0373580dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:44.868457 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182a25e-2cb8-455d-b9ea-aac0373580dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:44.870066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182a25e-2cb8-455d-b9ea-aac0373580dd admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:44.881293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1268/2554] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:32:44.885390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2405be6-4504-4753-98e2-1f2236f716a9 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:44.895740 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2405be6-4504-4753-98e2-1f2236f716a9 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:44.900259 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694cba2-48f7-4e45-9d48-91cf6effc06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:44.900891 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694cba2-48f7-4e45-9d48-91cf6effc06c admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:44.901194 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694cba2-48f7-4e45-9d48-91cf6effc06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:44.901471 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694cba2-48f7-4e45-9d48-91cf6effc06c admin admin] PATH_INFO: `/v3/projects/504d868b83b84e7ebe8ec15d24c8504a/users/073b588c191b49949a2b7508b0e99b28/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:44.903605 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:32:44.917632 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694cba2-48f7-4e45-9d48-91cf6effc06c admin admin] RBAC: Authorizing `identity:create_grant(project_id=504d868b83b84e7ebe8ec15d24c8504a, user_id=073b588c191b49949a2b7508b0e99b28, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:44.920270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694cba2-48f7-4e45-9d48-91cf6effc06c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:44.932490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1287/2555] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:32:44 2022] PUT /identity/v3/projects/504d868b83b84e7ebe8ec15d24c8504a/users/073b588c191b49949a2b7508b0e99b28/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:32:44.944293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aee580-b114-4204-8f9c-1314ffdc57f3 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:44.944584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aee580-b114-4204-8f9c-1314ffdc57f3 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:44.944858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aee580-b114-4204-8f9c-1314ffdc57f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:44.975088 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-22aee580-b114-4204-8f9c-1314ffdc57f3 None None] MFA Rules not processed for user `073b588c191b49949a2b7508b0e99b28`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:32:44.976358 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-22aee580-b114-4204-8f9c-1314ffdc57f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:45.040382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1269/2556] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:32:44 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:32:45.044779 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694cba2-48f7-4e45-9d48-91cf6effc06c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:45.106129 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694cba2-48f7-4e45-9d48-91cf6effc06c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:45.110785 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9c4cc-c7cf-4349-879a-13fde0fc4f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:45.112026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9c4cc-c7cf-4349-879a-13fde0fc4f3d admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:45.112445 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9c4cc-c7cf-4349-879a-13fde0fc4f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:45.112730 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9c4cc-c7cf-4349-879a-13fde0fc4f3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:45.113392 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e9c4cc-c7cf-4349-879a-13fde0fc4f3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:45.115276 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e9c4cc-c7cf-4349-879a-13fde0fc4f3d admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:45.141587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1288/2557] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:32:45 2022] POST /identity/v3/projects => generated 409 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:32:45.145937 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aee580-b114-4204-8f9c-1314ffdc57f3 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:45.156716 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aee580-b114-4204-8f9c-1314ffdc57f3 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:45.168362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:32:45.214803 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db11c4d-bacf-4b1f-a076-61edb22cef5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:45.215418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db11c4d-bacf-4b1f-a076-61edb22cef5b admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:45.215701 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db11c4d-bacf-4b1f-a076-61edb22cef5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:45.215979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db11c4d-bacf-4b1f-a076-61edb22cef5b admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:45.216595 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db11c4d-bacf-4b1f-a076-61edb22cef5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:45.218769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db11c4d-bacf-4b1f-a076-61edb22cef5b admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:45.240980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1270/2558] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:32:45 2022] POST /identity/v3/users => generated 311 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:32:45.245205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9c4cc-c7cf-4349-879a-13fde0fc4f3d admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:45.255984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9c4cc-c7cf-4349-879a-13fde0fc4f3d admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:45.260506 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4877c8-12b9-492d-b8a7-65a9acc8b045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:45.261096 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4877c8-12b9-492d-b8a7-65a9acc8b045 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:45.261404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4877c8-12b9-492d-b8a7-65a9acc8b045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:45.261679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4877c8-12b9-492d-b8a7-65a9acc8b045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:45.262304 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4877c8-12b9-492d-b8a7-65a9acc8b045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:45.263893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4877c8-12b9-492d-b8a7-65a9acc8b045 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:45.275786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1289/2559] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:32:45.279818 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db11c4d-bacf-4b1f-a076-61edb22cef5b admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:45.289872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db11c4d-bacf-4b1f-a076-61edb22cef5b admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:45.294514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1caff-8857-4244-a439-6e838fe879b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:45.295126 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1caff-8857-4244-a439-6e838fe879b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:45.295404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1caff-8857-4244-a439-6e838fe879b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:45.295673 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1caff-8857-4244-a439-6e838fe879b6 admin admin] PATH_INFO: `/v3/projects/bab1e55475b24efa8fb646aa10ddd606/users/713a3cdd5b104068844182992addf8c6/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:45.310607 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc1caff-8857-4244-a439-6e838fe879b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bab1e55475b24efa8fb646aa10ddd606, user_id=713a3cdd5b104068844182992addf8c6, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:45.313303 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc1caff-8857-4244-a439-6e838fe879b6 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:45.327237 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1271/2560] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:32:45 2022] PUT /identity/v3/projects/bab1e55475b24efa8fb646aa10ddd606/users/713a3cdd5b104068844182992addf8c6/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:32:45.331381 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4877c8-12b9-492d-b8a7-65a9acc8b045 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:45.341416 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4877c8-12b9-492d-b8a7-65a9acc8b045 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:45.397324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828630b5-94cc-4801-9a6b-ca11377c4bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:45.397962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828630b5-94cc-4801-9a6b-ca11377c4bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:45.398245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828630b5-94cc-4801-9a6b-ca11377c4bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:45.398521 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828630b5-94cc-4801-9a6b-ca11377c4bc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:45.399151 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828630b5-94cc-4801-9a6b-ca11377c4bc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:45.400733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828630b5-94cc-4801-9a6b-ca11377c4bc0 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:45.411134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1290/2561] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:32:45.415214 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1caff-8857-4244-a439-6e838fe879b6 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:45.426453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1caff-8857-4244-a439-6e838fe879b6 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:45.431095 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ba7c6-6a18-4ed1-8d47-63872b4e2f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:45.431745 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ba7c6-6a18-4ed1-8d47-63872b4e2f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:45.432030 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ba7c6-6a18-4ed1-8d47-63872b4e2f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:45.432306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ba7c6-6a18-4ed1-8d47-63872b4e2f6f admin admin] PATH_INFO: `/v3/projects/bab1e55475b24efa8fb646aa10ddd606/users/713a3cdd5b104068844182992addf8c6/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:45.434222 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ba7c6-6a18-4ed1-8d47-63872b4e2f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bab1e55475b24efa8fb646aa10ddd606, user_id=713a3cdd5b104068844182992addf8c6, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:45.436842 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ba7c6-6a18-4ed1-8d47-63872b4e2f6f admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:45.447237 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1272/2562] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:32:45 2022] PUT /identity/v3/projects/bab1e55475b24efa8fb646aa10ddd606/users/713a3cdd5b104068844182992addf8c6/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:32:45.458037 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410f5ec-67b2-494a-af38-6c99ac6a6ee2 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:45.458495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410f5ec-67b2-494a-af38-6c99ac6a6ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:45.458614 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410f5ec-67b2-494a-af38-6c99ac6a6ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:45.486238 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-0410f5ec-67b2-494a-af38-6c99ac6a6ee2 None None] MFA Rules not processed for user `713a3cdd5b104068844182992addf8c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:32:45.487924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-0410f5ec-67b2-494a-af38-6c99ac6a6ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:45.527129 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:32:45.555895 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1291/2563] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:32:45.566609 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ba7c6-6a18-4ed1-8d47-63872b4e2f6f admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:45.568421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e90ba7c6-6a18-4ed1-8d47-63872b4e2f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:45.627796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ba7c6-6a18-4ed1-8d47-63872b4e2f6f admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:45.631724 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80aa60-3486-4356-9e55-70b571041f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:45.632321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80aa60-3486-4356-9e55-70b571041f5d service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:45.632604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80aa60-3486-4356-9e55-70b571041f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:45.632874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80aa60-3486-4356-9e55-70b571041f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:45.642082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80aa60-3486-4356-9e55-70b571041f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:45.643632 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80aa60-3486-4356-9e55-70b571041f5d service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:45.653370 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1273/2564] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:32:45 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:32:45.803453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410f5ec-67b2-494a-af38-6c99ac6a6ee2 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:45.813008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0410f5ec-67b2-494a-af38-6c99ac6a6ee2 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:45.816474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d0ea0-148b-42fc-b6f0-58f5a98280b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:45.817070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d0ea0-148b-42fc-b6f0-58f5a98280b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:45.817380 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d0ea0-148b-42fc-b6f0-58f5a98280b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:45.817654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d0ea0-148b-42fc-b6f0-58f5a98280b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:45.874044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d0ea0-148b-42fc-b6f0-58f5a98280b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:45.875460 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d0ea0-148b-42fc-b6f0-58f5a98280b5 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:45.884701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1292/2565] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:32:45 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:32:55.246348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80aa60-3486-4356-9e55-70b571041f5d service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:55.308397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80aa60-3486-4356-9e55-70b571041f5d service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:55.312964 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb139c-e579-401f-bfa9-a77a06e7a107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:55.313629 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb139c-e579-401f-bfa9-a77a06e7a107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:55.313914 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb139c-e579-401f-bfa9-a77a06e7a107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:55.314189 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb139c-e579-401f-bfa9-a77a06e7a107 admin admin] PATH_INFO: `/v3/users/073b588c191b49949a2b7508b0e99b28` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:55.315404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb139c-e579-401f-bfa9-a77a06e7a107 admin admin] RBAC: Authorizing `identity:delete_user(user_id=073b588c191b49949a2b7508b0e99b28)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:55.318089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb139c-e579-401f-bfa9-a77a06e7a107 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:55.332734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:32:55.366033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f2cb139c-e579-401f-bfa9-a77a06e7a107 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '073b588c191b49949a2b7508b0e99b28'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:55.366612 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f2cb139c-e579-401f-bfa9-a77a06e7a107 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '073b588c191b49949a2b7508b0e99b28'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:55.378389 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f2cb139c-e579-401f-bfa9-a77a06e7a107 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '073b588c191b49949a2b7508b0e99b28'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:55.386111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f2cb139c-e579-401f-bfa9-a77a06e7a107 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '073b588c191b49949a2b7508b0e99b28'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:55.386720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f2cb139c-e579-401f-bfa9-a77a06e7a107 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '073b588c191b49949a2b7508b0e99b28'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:55.399497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1274/2566] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:32:55 2022] DELETE /identity/v3/users/073b588c191b49949a2b7508b0e99b28 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:32:55.403921 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea6510-6606-4071-8cab-1abcabb82527 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:55.404434 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea6510-6606-4071-8cab-1abcabb82527 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:55.404557 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea6510-6606-4071-8cab-1abcabb82527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:55.424648 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-27ea6510-6606-4071-8cab-1abcabb82527 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:32:55.425969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-27ea6510-6606-4071-8cab-1abcabb82527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:55.483347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1293/2567] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:32:55.493560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb139c-e579-401f-bfa9-a77a06e7a107 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:55.494974 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f2cb139c-e579-401f-bfa9-a77a06e7a107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:55.553195 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb139c-e579-401f-bfa9-a77a06e7a107 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:55.557171 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ed5d7-7737-4f48-a78a-5bf9523dbe52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:55.557807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ed5d7-7737-4f48-a78a-5bf9523dbe52 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:55.558096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ed5d7-7737-4f48-a78a-5bf9523dbe52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:55.558371 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57ed5d7-7737-4f48-a78a-5bf9523dbe52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:55.567939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ed5d7-7737-4f48-a78a-5bf9523dbe52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:55.569381 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57ed5d7-7737-4f48-a78a-5bf9523dbe52 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:55.578713 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1275/2568] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:32:55.853323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea6510-6606-4071-8cab-1abcabb82527 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:55.856164 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-27ea6510-6606-4071-8cab-1abcabb82527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:55.869061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea6510-6606-4071-8cab-1abcabb82527 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:55.874384 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:55.875069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:55.875350 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:55.875632 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28 admin admin] PATH_INFO: `/v3/projects/504d868b83b84e7ebe8ec15d24c8504a` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:55.876805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=504d868b83b84e7ebe8ec15d24c8504a)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:55.879281 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:55.934030 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '504d868b83b84e7ebe8ec15d24c8504a', 'request_id': 'req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:55.943605 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '504d868b83b84e7ebe8ec15d24c8504a', 'request_id': 'req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:55.953634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '504d868b83b84e7ebe8ec15d24c8504a', 'request_id': 'req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:55.962440 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:32:55.968401 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28 admin admin] The token cache is being invalidate because project 504d868b83b84e7ebe8ec15d24c8504a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:32:55.968896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:55.969846 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:55.972108 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1294/2569] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:32:55 2022] DELETE /identity/v3/projects/504d868b83b84e7ebe8ec15d24c8504a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:32:55.976554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ed5d7-7737-4f48-a78a-5bf9523dbe52 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:55.987026 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57ed5d7-7737-4f48-a78a-5bf9523dbe52 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:56.052087 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28adae76-22c2-4c00-887b-c1160b8ef1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:56.052716 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adae76-22c2-4c00-887b-c1160b8ef1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:56.053023 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adae76-22c2-4c00-887b-c1160b8ef1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:56.053382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28adae76-22c2-4c00-887b-c1160b8ef1b2 admin admin] PATH_INFO: `/v3/users/713a3cdd5b104068844182992addf8c6` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:56.054515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28adae76-22c2-4c00-887b-c1160b8ef1b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=713a3cdd5b104068844182992addf8c6)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:56.056732 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28adae76-22c2-4c00-887b-c1160b8ef1b2 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:56.097526 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:32:56.109100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-28adae76-22c2-4c00-887b-c1160b8ef1b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '713a3cdd5b104068844182992addf8c6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:56.109782 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-28adae76-22c2-4c00-887b-c1160b8ef1b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '713a3cdd5b104068844182992addf8c6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:56.122006 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-28adae76-22c2-4c00-887b-c1160b8ef1b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '713a3cdd5b104068844182992addf8c6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:56.130620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-28adae76-22c2-4c00-887b-c1160b8ef1b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '713a3cdd5b104068844182992addf8c6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:56.131263 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-28adae76-22c2-4c00-887b-c1160b8ef1b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '713a3cdd5b104068844182992addf8c6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:56.145876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1276/2570] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:32:55 2022] DELETE /identity/v3/users/713a3cdd5b104068844182992addf8c6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:32:56.440701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:56.443007 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:56.456271 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a9a72-f4fe-4a0c-9f31-c4d5064ded28 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:56.514766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123014e-c1f9-4b78-afd7-8dd253b199eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:56.515398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123014e-c1f9-4b78-afd7-8dd253b199eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:56.515678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123014e-c1f9-4b78-afd7-8dd253b199eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:56.515951 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123014e-c1f9-4b78-afd7-8dd253b199eb admin admin] PATH_INFO: `/v3/projects/bab1e55475b24efa8fb646aa10ddd606` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:56.517073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123014e-c1f9-4b78-afd7-8dd253b199eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bab1e55475b24efa8fb646aa10ddd606)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:56.519012 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123014e-c1f9-4b78-afd7-8dd253b199eb admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:56.567429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f123014e-c1f9-4b78-afd7-8dd253b199eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bab1e55475b24efa8fb646aa10ddd606', 'request_id': 'req-f123014e-c1f9-4b78-afd7-8dd253b199eb'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:56.574180 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f123014e-c1f9-4b78-afd7-8dd253b199eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bab1e55475b24efa8fb646aa10ddd606', 'request_id': 'req-f123014e-c1f9-4b78-afd7-8dd253b199eb'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:56.582071 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f123014e-c1f9-4b78-afd7-8dd253b199eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bab1e55475b24efa8fb646aa10ddd606', 'request_id': 'req-f123014e-c1f9-4b78-afd7-8dd253b199eb'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:56.593253 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f123014e-c1f9-4b78-afd7-8dd253b199eb admin admin] The token cache is being invalidate because project bab1e55475b24efa8fb646aa10ddd606 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:32:56.593582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f123014e-c1f9-4b78-afd7-8dd253b199eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:56.594234 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f123014e-c1f9-4b78-afd7-8dd253b199eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:32:56.595714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1295/2571] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:32:56 2022] DELETE /identity/v3/projects/bab1e55475b24efa8fb646aa10ddd606 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:32:56.601302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbe4d3-45bc-492c-9b90-3ec1c1872465 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:56.601795 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbe4d3-45bc-492c-9b90-3ec1c1872465 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:56.601921 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbe4d3-45bc-492c-9b90-3ec1c1872465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:56.621825 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-2bdbe4d3-45bc-492c-9b90-3ec1c1872465 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:32:56.623065 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2bdbe4d3-45bc-492c-9b90-3ec1c1872465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:56.681519 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1277/2572] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:32:56.689574 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662e5b8-fd3e-4b66-959c-07e3350f8b21 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:56.689855 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662e5b8-fd3e-4b66-959c-07e3350f8b21 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:56.690123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662e5b8-fd3e-4b66-959c-07e3350f8b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:56.709836 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-9662e5b8-fd3e-4b66-959c-07e3350f8b21 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:32:56.710944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9662e5b8-fd3e-4b66-959c-07e3350f8b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:56.718944 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1296/2573] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:32:56.723677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbe4d3-45bc-492c-9b90-3ec1c1872465 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:56.733606 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbe4d3-45bc-492c-9b90-3ec1c1872465 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:56.739465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b5e1f-07b2-49f7-bc35-06671149ee20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:56.740071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b5e1f-07b2-49f7-bc35-06671149ee20 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:56.740354 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b5e1f-07b2-49f7-bc35-06671149ee20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:56.740628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b5e1f-07b2-49f7-bc35-06671149ee20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:56.741310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b5e1f-07b2-49f7-bc35-06671149ee20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:32:56.741618 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b5e1f-07b2-49f7-bc35-06671149ee20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:56.743203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b5e1f-07b2-49f7-bc35-06671149ee20 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:56.754913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1278/2574] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:32:56.759115 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662e5b8-fd3e-4b66-959c-07e3350f8b21 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:56.768209 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662e5b8-fd3e-4b66-959c-07e3350f8b21 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:56.772808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d8ad1-d664-4693-963b-4d73dd8479c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:56.773437 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d8ad1-d664-4693-963b-4d73dd8479c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:56.773719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d8ad1-d664-4693-963b-4d73dd8479c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:56.773990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d8ad1-d664-4693-963b-4d73dd8479c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:56.774623 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d8ad1-d664-4693-963b-4d73dd8479c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:56.776487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d8ad1-d664-4693-963b-4d73dd8479c1 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:56.804258 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1297/2575] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:32:56 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:32:56.808365 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b5e1f-07b2-49f7-bc35-06671149ee20 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:56.818344 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b5e1f-07b2-49f7-bc35-06671149ee20 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:56.874679 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70084ee3-184e-4741-88d5-88c6fe94ca92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:56.875282 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70084ee3-184e-4741-88d5-88c6fe94ca92 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:56.875564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70084ee3-184e-4741-88d5-88c6fe94ca92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:56.875836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70084ee3-184e-4741-88d5-88c6fe94ca92 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:56.876449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70084ee3-184e-4741-88d5-88c6fe94ca92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:56.878383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70084ee3-184e-4741-88d5-88c6fe94ca92 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:56.900038 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1279/2576] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:32:56 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:32:56.903980 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d8ad1-d664-4693-963b-4d73dd8479c1 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:56.914718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d8ad1-d664-4693-963b-4d73dd8479c1 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:56.919272 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810b0c4-34b2-4b53-80fa-4b960ce10a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:56.919862 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810b0c4-34b2-4b53-80fa-4b960ce10a2f admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:56.920140 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810b0c4-34b2-4b53-80fa-4b960ce10a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:56.920411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810b0c4-34b2-4b53-80fa-4b960ce10a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:56.921035 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810b0c4-34b2-4b53-80fa-4b960ce10a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:56.922671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810b0c4-34b2-4b53-80fa-4b960ce10a2f admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:56.933034 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1298/2577] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:32:56.936796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70084ee3-184e-4741-88d5-88c6fe94ca92 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:56.946870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70084ee3-184e-4741-88d5-88c6fe94ca92 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:56.951664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ac851-c700-461d-9be2-00d0d1602ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:56.952254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ac851-c700-461d-9be2-00d0d1602ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:56.952538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ac851-c700-461d-9be2-00d0d1602ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:56.952811 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ac851-c700-461d-9be2-00d0d1602ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:56.953480 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ac851-c700-461d-9be2-00d0d1602ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:56.955072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ac851-c700-461d-9be2-00d0d1602ce3 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:56.965567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1280/2578] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:32:56.969785 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810b0c4-34b2-4b53-80fa-4b960ce10a2f admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:56.980261 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810b0c4-34b2-4b53-80fa-4b960ce10a2f admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:56.984864 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e340e-56e1-4564-b380-9b4f475e6bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:56.985557 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e340e-56e1-4564-b380-9b4f475e6bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:56.985841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e340e-56e1-4564-b380-9b4f475e6bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:56.986119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e340e-56e1-4564-b380-9b4f475e6bc0 admin admin] PATH_INFO: `/v3/projects/d980ea3aa07c4f619868144c9d5e5a5f/users/286d8137b22949c197c2bcf75bc7e2fa/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:57.001737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e340e-56e1-4564-b380-9b4f475e6bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d980ea3aa07c4f619868144c9d5e5a5f, user_id=286d8137b22949c197c2bcf75bc7e2fa, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:57.004360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e340e-56e1-4564-b380-9b4f475e6bc0 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:57.017007 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1299/2579] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:32:56 2022] PUT /identity/v3/projects/d980ea3aa07c4f619868144c9d5e5a5f/users/286d8137b22949c197c2bcf75bc7e2fa/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:32:57.028917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63646738-62de-4dd0-b142-50083845c869 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:57.029246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63646738-62de-4dd0-b142-50083845c869 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:57.029525 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63646738-62de-4dd0-b142-50083845c869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:57.056976 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-63646738-62de-4dd0-b142-50083845c869 None None] MFA Rules not processed for user `286d8137b22949c197c2bcf75bc7e2fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:32:57.058204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-63646738-62de-4dd0-b142-50083845c869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:57.063153 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:32:57.123368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1281/2580] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:32:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:32:57.134713 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e340e-56e1-4564-b380-9b4f475e6bc0 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:32:57.136356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-8a6e340e-56e1-4564-b380-9b4f475e6bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:32:57.166370 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:32:57.197269 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e340e-56e1-4564-b380-9b4f475e6bc0 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:32:57.200873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a6b66-62d0-49e1-afef-a5ed5a0b9f4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:32:57.201514 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a6b66-62d0-49e1-afef-a5ed5a0b9f4e service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:32:57.201797 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a6b66-62d0-49e1-afef-a5ed5a0b9f4e service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:32:57.202069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a6b66-62d0-49e1-afef-a5ed5a0b9f4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:32:57.210398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a6b66-62d0-49e1-afef-a5ed5a0b9f4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:32:57.211806 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a6b66-62d0-49e1-afef-a5ed5a0b9f4e service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:32:57.220676 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1300/2581] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:32:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:33:05.604984 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63646738-62de-4dd0-b142-50083845c869 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:05.668572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63646738-62de-4dd0-b142-50083845c869 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:05.673113 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789697f2-e69a-437b-945e-8ff5ec39e4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:05.673776 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789697f2-e69a-437b-945e-8ff5ec39e4a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:05.674062 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789697f2-e69a-437b-945e-8ff5ec39e4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:05.674338 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789697f2-e69a-437b-945e-8ff5ec39e4a2 admin admin] PATH_INFO: `/v3/users/286d8137b22949c197c2bcf75bc7e2fa` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:05.675471 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789697f2-e69a-437b-945e-8ff5ec39e4a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=286d8137b22949c197c2bcf75bc7e2fa)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:05.677898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789697f2-e69a-437b-945e-8ff5ec39e4a2 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:05.724240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-789697f2-e69a-437b-945e-8ff5ec39e4a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '286d8137b22949c197c2bcf75bc7e2fa'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:05.725139 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-789697f2-e69a-437b-945e-8ff5ec39e4a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '286d8137b22949c197c2bcf75bc7e2fa'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:05.737876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-789697f2-e69a-437b-945e-8ff5ec39e4a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '286d8137b22949c197c2bcf75bc7e2fa'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:05.745009 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-789697f2-e69a-437b-945e-8ff5ec39e4a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '286d8137b22949c197c2bcf75bc7e2fa'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:05.745704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-789697f2-e69a-437b-945e-8ff5ec39e4a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '286d8137b22949c197c2bcf75bc7e2fa'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:05.758343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1282/2582] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:33:05 2022] DELETE /identity/v3/users/286d8137b22949c197c2bcf75bc7e2fa => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:05.763027 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229c01f-a9f4-46ed-97b9-63ef08be1000 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:05.763507 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229c01f-a9f4-46ed-97b9-63ef08be1000 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:05.763618 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7229c01f-a9f4-46ed-97b9-63ef08be1000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:05.787760 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7229c01f-a9f4-46ed-97b9-63ef08be1000 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:05.788964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7229c01f-a9f4-46ed-97b9-63ef08be1000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:05.850817 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1301/2583] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:05.860293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789697f2-e69a-437b-945e-8ff5ec39e4a2 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:05.861697 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-789697f2-e69a-437b-945e-8ff5ec39e4a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:05.892311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:05.920161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789697f2-e69a-437b-945e-8ff5ec39e4a2 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:05.923914 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94a938-1d40-4a91-ab93-cbaaeeb4e799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:05.924512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94a938-1d40-4a91-ab93-cbaaeeb4e799 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:05.924788 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94a938-1d40-4a91-ab93-cbaaeeb4e799 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:05.925059 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94a938-1d40-4a91-ab93-cbaaeeb4e799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:05.934064 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94a938-1d40-4a91-ab93-cbaaeeb4e799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:05.935477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94a938-1d40-4a91-ab93-cbaaeeb4e799 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:05.945382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1283/2584] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:06.213896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229c01f-a9f4-46ed-97b9-63ef08be1000 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:06.215288 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7229c01f-a9f4-46ed-97b9-63ef08be1000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:06.232919 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7229c01f-a9f4-46ed-97b9-63ef08be1000 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:06.245086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1322375-8afc-497a-89ec-5367193d2fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:06.245763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1322375-8afc-497a-89ec-5367193d2fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:06.246041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1322375-8afc-497a-89ec-5367193d2fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:06.246313 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1322375-8afc-497a-89ec-5367193d2fa2 admin admin] PATH_INFO: `/v3/projects/d980ea3aa07c4f619868144c9d5e5a5f` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:06.247418 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1322375-8afc-497a-89ec-5367193d2fa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d980ea3aa07c4f619868144c9d5e5a5f)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:06.249368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1322375-8afc-497a-89ec-5367193d2fa2 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:06.295431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b1322375-8afc-497a-89ec-5367193d2fa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd980ea3aa07c4f619868144c9d5e5a5f', 'request_id': 'req-b1322375-8afc-497a-89ec-5367193d2fa2'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:06.302314 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b1322375-8afc-497a-89ec-5367193d2fa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd980ea3aa07c4f619868144c9d5e5a5f', 'request_id': 'req-b1322375-8afc-497a-89ec-5367193d2fa2'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:06.309995 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b1322375-8afc-497a-89ec-5367193d2fa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd980ea3aa07c4f619868144c9d5e5a5f', 'request_id': 'req-b1322375-8afc-497a-89ec-5367193d2fa2'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:06.320710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b1322375-8afc-497a-89ec-5367193d2fa2 admin admin] The token cache is being invalidate because project d980ea3aa07c4f619868144c9d5e5a5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:33:06.321000 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b1322375-8afc-497a-89ec-5367193d2fa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:06.321656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b1322375-8afc-497a-89ec-5367193d2fa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:06.323126 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1302/2585] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:33:06 2022] DELETE /identity/v3/projects/d980ea3aa07c4f619868144c9d5e5a5f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:06.328065 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac96b11-bf6d-4856-94ff-dded14274ec4 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:06.328359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac96b11-bf6d-4856-94ff-dded14274ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:06.328639 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac96b11-bf6d-4856-94ff-dded14274ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:06.349575 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-aac96b11-bf6d-4856-94ff-dded14274ec4 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:06.350786 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-aac96b11-bf6d-4856-94ff-dded14274ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:06.412490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1284/2586] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:06.420200 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f96dc2-8262-4ee8-b8f2-0f1cd661acb9 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:06.420494 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f96dc2-8262-4ee8-b8f2-0f1cd661acb9 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:06.420771 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f96dc2-8262-4ee8-b8f2-0f1cd661acb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:06.440864 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-54f96dc2-8262-4ee8-b8f2-0f1cd661acb9 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:06.442129 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-54f96dc2-8262-4ee8-b8f2-0f1cd661acb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:06.449890 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1303/2587] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:06.454161 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac96b11-bf6d-4856-94ff-dded14274ec4 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:06.464897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac96b11-bf6d-4856-94ff-dded14274ec4 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:06.469775 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b145f8c-1980-4b48-99be-13551fd6d3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:06.470375 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b145f8c-1980-4b48-99be-13551fd6d3ab admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:06.470658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b145f8c-1980-4b48-99be-13551fd6d3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:06.470935 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b145f8c-1980-4b48-99be-13551fd6d3ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:06.471581 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b145f8c-1980-4b48-99be-13551fd6d3ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:33:06.471887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b145f8c-1980-4b48-99be-13551fd6d3ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:06.474332 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b145f8c-1980-4b48-99be-13551fd6d3ab admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:06.485913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1285/2588] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:33:06.490764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f96dc2-8262-4ee8-b8f2-0f1cd661acb9 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:06.500071 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f96dc2-8262-4ee8-b8f2-0f1cd661acb9 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:06.504781 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc755269-8d59-415c-a709-c083736cdd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:06.505404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc755269-8d59-415c-a709-c083736cdd60 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:06.505690 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc755269-8d59-415c-a709-c083736cdd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:06.505967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc755269-8d59-415c-a709-c083736cdd60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:06.506607 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc755269-8d59-415c-a709-c083736cdd60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:06.508480 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc755269-8d59-415c-a709-c083736cdd60 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:06.534490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1304/2589] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:33:06 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:06.539299 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b145f8c-1980-4b48-99be-13551fd6d3ab admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:06.548998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b145f8c-1980-4b48-99be-13551fd6d3ab admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:06.604078 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3395ac3-764d-424f-8ca3-3484c02de5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:06.604674 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3395ac3-764d-424f-8ca3-3484c02de5bd admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:06.604953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3395ac3-764d-424f-8ca3-3484c02de5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:06.605254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3395ac3-764d-424f-8ca3-3484c02de5bd admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:06.605854 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3395ac3-764d-424f-8ca3-3484c02de5bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:06.607772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3395ac3-764d-424f-8ca3-3484c02de5bd admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:06.629057 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1286/2590] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:33:06 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:06.633266 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc755269-8d59-415c-a709-c083736cdd60 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:06.643639 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc755269-8d59-415c-a709-c083736cdd60 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:06.648523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e63cf-34f4-40b2-98d8-8462d98e6c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:06.649170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e63cf-34f4-40b2-98d8-8462d98e6c60 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:06.649460 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e63cf-34f4-40b2-98d8-8462d98e6c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:06.649733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e63cf-34f4-40b2-98d8-8462d98e6c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:06.650362 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e63cf-34f4-40b2-98d8-8462d98e6c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:06.651934 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e63cf-34f4-40b2-98d8-8462d98e6c60 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:06.662109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1305/2591] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:06.665956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3395ac3-764d-424f-8ca3-3484c02de5bd admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:06.675632 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3395ac3-764d-424f-8ca3-3484c02de5bd admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:06.681164 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db04360a-d079-4731-a04d-6dfe4868b44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:06.681763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04360a-d079-4731-a04d-6dfe4868b44d admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:06.682058 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04360a-d079-4731-a04d-6dfe4868b44d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:06.682318 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04360a-d079-4731-a04d-6dfe4868b44d admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:06.682948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db04360a-d079-4731-a04d-6dfe4868b44d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:06.684534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db04360a-d079-4731-a04d-6dfe4868b44d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:06.694920 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1287/2592] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:06.698918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e63cf-34f4-40b2-98d8-8462d98e6c60 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:06.709171 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e63cf-34f4-40b2-98d8-8462d98e6c60 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:06.713691 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065a7ae-3f20-4427-9512-514e626a548c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:06.714333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065a7ae-3f20-4427-9512-514e626a548c admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:06.714613 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065a7ae-3f20-4427-9512-514e626a548c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:06.714887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065a7ae-3f20-4427-9512-514e626a548c admin admin] PATH_INFO: `/v3/projects/6794d610d33f444685787130b811a83a/users/285093a038b447cb8b64495e20d0339a/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:06.726410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:06.730938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065a7ae-3f20-4427-9512-514e626a548c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6794d610d33f444685787130b811a83a, user_id=285093a038b447cb8b64495e20d0339a, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:06.734307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065a7ae-3f20-4427-9512-514e626a548c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:06.746965 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1306/2593] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:06 2022] PUT /identity/v3/projects/6794d610d33f444685787130b811a83a/users/285093a038b447cb8b64495e20d0339a/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:06.758332 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac29c2-1bf8-4da2-8450-072b449bf57d None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:06.758619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac29c2-1bf8-4da2-8450-072b449bf57d None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:06.758900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac29c2-1bf8-4da2-8450-072b449bf57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:06.787845 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-a1ac29c2-1bf8-4da2-8450-072b449bf57d None None] MFA Rules not processed for user `285093a038b447cb8b64495e20d0339a`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:06.789060 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-a1ac29c2-1bf8-4da2-8450-072b449bf57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:06.812544 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:06.854348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1288/2594] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:06.858811 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065a7ae-3f20-4427-9512-514e626a548c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:06.919594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065a7ae-3f20-4427-9512-514e626a548c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:06.924125 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f711a2-e035-420d-940a-4beb673b2c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:06.924740 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f711a2-e035-420d-940a-4beb673b2c7c admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:06.925006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f711a2-e035-420d-940a-4beb673b2c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:06.925306 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f711a2-e035-420d-940a-4beb673b2c7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:06.925928 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f711a2-e035-420d-940a-4beb673b2c7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:06.928172 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f711a2-e035-420d-940a-4beb673b2c7c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:06.955061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1307/2595] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:33:06 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:06.959744 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac29c2-1bf8-4da2-8450-072b449bf57d None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:06.976908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac29c2-1bf8-4da2-8450-072b449bf57d None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:07.035422 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db5e29-38cd-4e9c-a99f-03de7a329b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:07.036643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db5e29-38cd-4e9c-a99f-03de7a329b7c admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:07.036923 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db5e29-38cd-4e9c-a99f-03de7a329b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:07.037245 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db5e29-38cd-4e9c-a99f-03de7a329b7c admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:07.037908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db5e29-38cd-4e9c-a99f-03de7a329b7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:07.039859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db5e29-38cd-4e9c-a99f-03de7a329b7c admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:07.061266 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1289/2596] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:33:06 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:33:07.065303 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f711a2-e035-420d-940a-4beb673b2c7c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:07.076083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f711a2-e035-420d-940a-4beb673b2c7c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:07.080743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c311305-e6fb-4d83-83bd-a9d3b56dc885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:07.081413 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c311305-e6fb-4d83-83bd-a9d3b56dc885 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:07.081682 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c311305-e6fb-4d83-83bd-a9d3b56dc885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:07.081949 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c311305-e6fb-4d83-83bd-a9d3b56dc885 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:07.083392 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c311305-e6fb-4d83-83bd-a9d3b56dc885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:07.085091 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c311305-e6fb-4d83-83bd-a9d3b56dc885 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:07.096145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1308/2597] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:07.100313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db5e29-38cd-4e9c-a99f-03de7a329b7c admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:07.110445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db5e29-38cd-4e9c-a99f-03de7a329b7c admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:07.115787 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f490dc-8f36-4f6d-b17c-3c885df40ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:07.116394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f490dc-8f36-4f6d-b17c-3c885df40ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:07.116677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f490dc-8f36-4f6d-b17c-3c885df40ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:07.116953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f490dc-8f36-4f6d-b17c-3c885df40ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:07.117777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f490dc-8f36-4f6d-b17c-3c885df40ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:07.119402 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f490dc-8f36-4f6d-b17c-3c885df40ad9 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:07.132310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1290/2598] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:07.135585 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c311305-e6fb-4d83-83bd-a9d3b56dc885 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:07.146655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c311305-e6fb-4d83-83bd-a9d3b56dc885 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:07.151761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ee434-a297-4798-8380-329270a16cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:07.152406 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ee434-a297-4798-8380-329270a16cf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:07.152801 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ee434-a297-4798-8380-329270a16cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:07.153080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ee434-a297-4798-8380-329270a16cf9 admin admin] PATH_INFO: `/v3/projects/250a212af3c9448b84bb5b049b604942/users/afcb7d6a35674119962bcf4922ab7951/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:07.168368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ee434-a297-4798-8380-329270a16cf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=250a212af3c9448b84bb5b049b604942, user_id=afcb7d6a35674119962bcf4922ab7951, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:07.171084 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ee434-a297-4798-8380-329270a16cf9 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:07.186079 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1309/2599] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:07 2022] PUT /identity/v3/projects/250a212af3c9448b84bb5b049b604942/users/afcb7d6a35674119962bcf4922ab7951/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:07.197352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57721b0e-5a61-4396-879f-4da8b79a7ace None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:07.197645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57721b0e-5a61-4396-879f-4da8b79a7ace None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:07.197923 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57721b0e-5a61-4396-879f-4da8b79a7ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:07.230838 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-57721b0e-5a61-4396-879f-4da8b79a7ace None None] MFA Rules not processed for user `afcb7d6a35674119962bcf4922ab7951`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:07.232182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-57721b0e-5a61-4396-879f-4da8b79a7ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:07.280074 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ee434-a297-4798-8380-329270a16cf9 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:07.281839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9f0ee434-a297-4798-8380-329270a16cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:07.300438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1291/2600] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:07.305543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57721b0e-5a61-4396-879f-4da8b79a7ace None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:07.339455 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:07.340675 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ee434-a297-4798-8380-329270a16cf9 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:07.344699 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4afbac-6672-4db6-8896-5c72039b5a6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:07.345349 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4afbac-6672-4db6-8896-5c72039b5a6d service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:07.345637 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4afbac-6672-4db6-8896-5c72039b5a6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:07.345910 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4afbac-6672-4db6-8896-5c72039b5a6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:07.347911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5f4afbac-6672-4db6-8896-5c72039b5a6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:07.371109 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57721b0e-5a61-4396-879f-4da8b79a7ace None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:07.375800 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0571a2e-5afe-4a69-8865-8cd28173306e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:07.376407 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0571a2e-5afe-4a69-8865-8cd28173306e admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:07.376692 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0571a2e-5afe-4a69-8865-8cd28173306e admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:07.376967 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0571a2e-5afe-4a69-8865-8cd28173306e admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:07.377684 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0571a2e-5afe-4a69-8865-8cd28173306e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:07.379799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0571a2e-5afe-4a69-8865-8cd28173306e admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:07.401411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:07.407833 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1292/2601] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:33:07 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:33:07.412557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0571a2e-5afe-4a69-8865-8cd28173306e admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:07.427088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4afbac-6672-4db6-8896-5c72039b5a6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:07.428575 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4afbac-6672-4db6-8896-5c72039b5a6d service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:07.439537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1310/2602] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:33:07.477675 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0571a2e-5afe-4a69-8865-8cd28173306e admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:07.483515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e389495-93da-4661-b8a2-f99457368b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:07.484135 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e389495-93da-4661-b8a2-f99457368b4e admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:07.484417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e389495-93da-4661-b8a2-f99457368b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:07.484688 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e389495-93da-4661-b8a2-f99457368b4e admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:07.485358 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e389495-93da-4661-b8a2-f99457368b4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:07.487275 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e389495-93da-4661-b8a2-f99457368b4e admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:07.509670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1293/2603] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:33:07 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:33:07.515051 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4afbac-6672-4db6-8896-5c72039b5a6d service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:07.526482 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4afbac-6672-4db6-8896-5c72039b5a6d service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:07.531815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d0658-2b8c-4655-8b9e-c22f1b8915ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:07.532152 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d0658-2b8c-4655-8b9e-c22f1b8915ec admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:07.532441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d0658-2b8c-4655-8b9e-c22f1b8915ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:07.532724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d0658-2b8c-4655-8b9e-c22f1b8915ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:07.533404 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d0658-2b8c-4655-8b9e-c22f1b8915ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:07.535902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d0658-2b8c-4655-8b9e-c22f1b8915ec admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:07.546321 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1311/2604] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:33:07.550357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e389495-93da-4661-b8a2-f99457368b4e admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:07.561255 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e389495-93da-4661-b8a2-f99457368b4e admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:07.565940 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c3d93-5d8d-4b7d-a587-a13280ad64d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:07.566596 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c3d93-5d8d-4b7d-a587-a13280ad64d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:07.567643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c3d93-5d8d-4b7d-a587-a13280ad64d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:07.568127 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c3d93-5d8d-4b7d-a587-a13280ad64d8 admin admin] PATH_INFO: `/v3/projects/80e87d5ed2e24caa9571a63981733b46/users/eedac78a9101497e98e0b3264bfd7055/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:07.583659 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c3d93-5d8d-4b7d-a587-a13280ad64d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e87d5ed2e24caa9571a63981733b46, user_id=eedac78a9101497e98e0b3264bfd7055, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:07.586354 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c3d93-5d8d-4b7d-a587-a13280ad64d8 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:07.601617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1294/2605] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:07 2022] PUT /identity/v3/projects/80e87d5ed2e24caa9571a63981733b46/users/eedac78a9101497e98e0b3264bfd7055/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:07.605531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d0658-2b8c-4655-8b9e-c22f1b8915ec admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:07.616115 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d0658-2b8c-4655-8b9e-c22f1b8915ec admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:07.792624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebac428-ef75-47be-b021-3054e4ca47a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:07.793117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebac428-ef75-47be-b021-3054e4ca47a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:07.793422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebac428-ef75-47be-b021-3054e4ca47a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:07.793687 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebac428-ef75-47be-b021-3054e4ca47a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:07.794307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebac428-ef75-47be-b021-3054e4ca47a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:07.795875 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebac428-ef75-47be-b021-3054e4ca47a2 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:07.807368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1312/2606] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:07.811727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c3d93-5d8d-4b7d-a587-a13280ad64d8 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:07.822582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c3d93-5d8d-4b7d-a587-a13280ad64d8 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:07.827119 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a41e20-0e5b-4ef3-bd78-dbd3503ce98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:07.827761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a41e20-0e5b-4ef3-bd78-dbd3503ce98e admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:07.828042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a41e20-0e5b-4ef3-bd78-dbd3503ce98e admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:07.828316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a41e20-0e5b-4ef3-bd78-dbd3503ce98e admin admin] PATH_INFO: `/v3/projects/80e87d5ed2e24caa9571a63981733b46/users/eedac78a9101497e98e0b3264bfd7055/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:07.830223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a41e20-0e5b-4ef3-bd78-dbd3503ce98e admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e87d5ed2e24caa9571a63981733b46, user_id=eedac78a9101497e98e0b3264bfd7055, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:07.833732 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a41e20-0e5b-4ef3-bd78-dbd3503ce98e admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:07.848725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1295/2607] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:07 2022] PUT /identity/v3/projects/80e87d5ed2e24caa9571a63981733b46/users/eedac78a9101497e98e0b3264bfd7055/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:07.858818 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e36e0f-180f-4bf8-9e0a-53a6a46dac85 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:07.859296 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e36e0f-180f-4bf8-9e0a-53a6a46dac85 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:07.859394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e36e0f-180f-4bf8-9e0a-53a6a46dac85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:07.886625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-76e36e0f-180f-4bf8-9e0a-53a6a46dac85 None None] MFA Rules not processed for user `eedac78a9101497e98e0b3264bfd7055`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:07.887804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-76e36e0f-180f-4bf8-9e0a-53a6a46dac85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:07.961051 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1313/2608] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:07.972513 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a41e20-0e5b-4ef3-bd78-dbd3503ce98e admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:07.974512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-13a41e20-0e5b-4ef3-bd78-dbd3503ce98e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:08.032207 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a41e20-0e5b-4ef3-bd78-dbd3503ce98e admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:08.036008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e2b44-3b8b-4a4d-b728-b7064b6542df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:08.036598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e2b44-3b8b-4a4d-b728-b7064b6542df service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:08.036879 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e2b44-3b8b-4a4d-b728-b7064b6542df service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:08.037182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e2b44-3b8b-4a4d-b728-b7064b6542df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:08.060703 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:08.093808 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e2b44-3b8b-4a4d-b728-b7064b6542df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:08.095233 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e2b44-3b8b-4a4d-b728-b7064b6542df service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:08.104240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1296/2609] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:08.628426 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e36e0f-180f-4bf8-9e0a-53a6a46dac85 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:08.629825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-76e36e0f-180f-4bf8-9e0a-53a6a46dac85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:08.685755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e36e0f-180f-4bf8-9e0a-53a6a46dac85 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:08.689262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe120411-5b88-4e57-9ce9-3ca7333f9e55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:08.689861 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe120411-5b88-4e57-9ce9-3ca7333f9e55 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:08.690134 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe120411-5b88-4e57-9ce9-3ca7333f9e55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:08.690402 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe120411-5b88-4e57-9ce9-3ca7333f9e55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:08.692103 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-fe120411-5b88-4e57-9ce9-3ca7333f9e55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:08.698603 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:08.763604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe120411-5b88-4e57-9ce9-3ca7333f9e55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:08.765020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe120411-5b88-4e57-9ce9-3ca7333f9e55 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:08.776583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1314/2610] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:09.616738 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e2b44-3b8b-4a4d-b728-b7064b6542df service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:09.618274 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-dc3e2b44-3b8b-4a4d-b728-b7064b6542df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:09.633069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e2b44-3b8b-4a4d-b728-b7064b6542df service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:09.637049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7874f1e-8506-4a1b-8ae1-105f65543340 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:09.638046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7874f1e-8506-4a1b-8ae1-105f65543340 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:09.638329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7874f1e-8506-4a1b-8ae1-105f65543340 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:09.638605 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7874f1e-8506-4a1b-8ae1-105f65543340 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:09.696741 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7874f1e-8506-4a1b-8ae1-105f65543340 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:09.698220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7874f1e-8506-4a1b-8ae1-105f65543340 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:09.707357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1297/2611] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:10.361401 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe120411-5b88-4e57-9ce9-3ca7333f9e55 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:10.371274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe120411-5b88-4e57-9ce9-3ca7333f9e55 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:10.375019 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41476fa7-cb98-414f-a22f-c271f8c4a556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:10.375613 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41476fa7-cb98-414f-a22f-c271f8c4a556 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:10.376057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41476fa7-cb98-414f-a22f-c271f8c4a556 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:10.376353 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41476fa7-cb98-414f-a22f-c271f8c4a556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:10.385766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41476fa7-cb98-414f-a22f-c271f8c4a556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:10.387203 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41476fa7-cb98-414f-a22f-c271f8c4a556 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:10.397257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1315/2612] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:14.195357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7874f1e-8506-4a1b-8ae1-105f65543340 service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:14.204642 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7874f1e-8506-4a1b-8ae1-105f65543340 service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:14.208279 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d096d0f-62b0-48e2-ab72-974e44767efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:14.209426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d096d0f-62b0-48e2-ab72-974e44767efc service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:14.209753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d096d0f-62b0-48e2-ab72-974e44767efc service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:14.210041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d096d0f-62b0-48e2-ab72-974e44767efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:14.211754 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-3d096d0f-62b0-48e2-ab72-974e44767efc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:14.277228 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d096d0f-62b0-48e2-ab72-974e44767efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:14.278676 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d096d0f-62b0-48e2-ab72-974e44767efc service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:14.289737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1298/2613] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:14.311158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41476fa7-cb98-414f-a22f-c271f8c4a556 service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:14.321660 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41476fa7-cb98-414f-a22f-c271f8c4a556 service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:14.326144 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766a310-66d4-4717-b6ca-2753967cf106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:14.326765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766a310-66d4-4717-b6ca-2753967cf106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:14.327042 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766a310-66d4-4717-b6ca-2753967cf106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:14.327314 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766a310-66d4-4717-b6ca-2753967cf106 admin admin] PATH_INFO: `/v3/users/285093a038b447cb8b64495e20d0339a` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:14.328410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766a310-66d4-4717-b6ca-2753967cf106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=285093a038b447cb8b64495e20d0339a)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:14.330344 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766a310-66d4-4717-b6ca-2753967cf106 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:14.380361 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d766a310-66d4-4717-b6ca-2753967cf106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '285093a038b447cb8b64495e20d0339a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:14.381070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d766a310-66d4-4717-b6ca-2753967cf106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '285093a038b447cb8b64495e20d0339a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:14.389026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d766a310-66d4-4717-b6ca-2753967cf106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '285093a038b447cb8b64495e20d0339a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:14.389818 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d766a310-66d4-4717-b6ca-2753967cf106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '285093a038b447cb8b64495e20d0339a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:14.397331 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d766a310-66d4-4717-b6ca-2753967cf106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '285093a038b447cb8b64495e20d0339a'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:14.415077 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1316/2614] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:33:14 2022] DELETE /identity/v3/users/285093a038b447cb8b64495e20d0339a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:14.419656 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dc43a-c449-43b0-9504-bf2e5cd94366 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:14.419950 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dc43a-c449-43b0-9504-bf2e5cd94366 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:14.420225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dc43a-c449-43b0-9504-bf2e5cd94366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:14.440007 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-1f7dc43a-c449-43b0-9504-bf2e5cd94366 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:14.441238 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1f7dc43a-c449-43b0-9504-bf2e5cd94366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:14.493071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:14.502889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1299/2615] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:33:14.512525 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766a310-66d4-4717-b6ca-2753967cf106 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:14.514009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-d766a310-66d4-4717-b6ca-2753967cf106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:14.569841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766a310-66d4-4717-b6ca-2753967cf106 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:14.573474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3683aa-6170-4781-b412-46b51fcde86f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:14.574077 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3683aa-6170-4781-b412-46b51fcde86f service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:14.574356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3683aa-6170-4781-b412-46b51fcde86f service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:14.574625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3683aa-6170-4781-b412-46b51fcde86f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:14.580422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:14.585501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3683aa-6170-4781-b412-46b51fcde86f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:14.587423 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3683aa-6170-4781-b412-46b51fcde86f service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:14.598558 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1317/2616] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:14.867496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7dc43a-c449-43b0-9504-bf2e5cd94366 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:14.869252 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1f7dc43a-c449-43b0-9504-bf2e5cd94366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:14.882645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7dc43a-c449-43b0-9504-bf2e5cd94366 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:14.887296 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721c669-e952-46e5-b3b6-88c363f13528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:14.887932 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721c669-e952-46e5-b3b6-88c363f13528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:14.888212 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721c669-e952-46e5-b3b6-88c363f13528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:14.888491 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721c669-e952-46e5-b3b6-88c363f13528 admin admin] PATH_INFO: `/v3/projects/6794d610d33f444685787130b811a83a` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:14.889634 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6721c669-e952-46e5-b3b6-88c363f13528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6794d610d33f444685787130b811a83a)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:14.891535 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6721c669-e952-46e5-b3b6-88c363f13528 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:14.940439 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6721c669-e952-46e5-b3b6-88c363f13528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6794d610d33f444685787130b811a83a', 'request_id': 'req-6721c669-e952-46e5-b3b6-88c363f13528'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:14.947347 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6721c669-e952-46e5-b3b6-88c363f13528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6794d610d33f444685787130b811a83a', 'request_id': 'req-6721c669-e952-46e5-b3b6-88c363f13528'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:14.956359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6721c669-e952-46e5-b3b6-88c363f13528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6794d610d33f444685787130b811a83a', 'request_id': 'req-6721c669-e952-46e5-b3b6-88c363f13528'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:14.969453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6721c669-e952-46e5-b3b6-88c363f13528 admin admin] The token cache is being invalidate because project 6794d610d33f444685787130b811a83a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:33:14.969753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6721c669-e952-46e5-b3b6-88c363f13528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:14.970412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6721c669-e952-46e5-b3b6-88c363f13528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:14.971930 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1300/2617] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:33:14 2022] DELETE /identity/v3/projects/6794d610d33f444685787130b811a83a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:14.976700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3683aa-6170-4781-b412-46b51fcde86f service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:14.986398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3683aa-6170-4781-b412-46b51fcde86f service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:15.043860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355042a-96d7-43be-8349-9181bde5a727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:15.044492 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355042a-96d7-43be-8349-9181bde5a727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:15.044773 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355042a-96d7-43be-8349-9181bde5a727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:15.045047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355042a-96d7-43be-8349-9181bde5a727 admin admin] PATH_INFO: `/v3/users/afcb7d6a35674119962bcf4922ab7951` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:15.046195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355042a-96d7-43be-8349-9181bde5a727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afcb7d6a35674119962bcf4922ab7951)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:15.048094 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355042a-96d7-43be-8349-9181bde5a727 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:15.096828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f355042a-96d7-43be-8349-9181bde5a727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afcb7d6a35674119962bcf4922ab7951'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:15.097528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f355042a-96d7-43be-8349-9181bde5a727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afcb7d6a35674119962bcf4922ab7951'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:15.105517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f355042a-96d7-43be-8349-9181bde5a727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afcb7d6a35674119962bcf4922ab7951'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:15.106139 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f355042a-96d7-43be-8349-9181bde5a727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afcb7d6a35674119962bcf4922ab7951'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:15.113321 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f355042a-96d7-43be-8349-9181bde5a727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afcb7d6a35674119962bcf4922ab7951'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:15.131854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1318/2618] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:33:14 2022] DELETE /identity/v3/users/afcb7d6a35674119962bcf4922ab7951 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:15.460988 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721c669-e952-46e5-b3b6-88c363f13528 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:15.463107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-6721c669-e952-46e5-b3b6-88c363f13528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:15.482361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6721c669-e952-46e5-b3b6-88c363f13528 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:15.549826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd162317-3840-4773-955e-4b1b93fd0e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:15.550471 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd162317-3840-4773-955e-4b1b93fd0e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:15.550756 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd162317-3840-4773-955e-4b1b93fd0e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:15.551036 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd162317-3840-4773-955e-4b1b93fd0e0f admin admin] PATH_INFO: `/v3/projects/250a212af3c9448b84bb5b049b604942` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:15.552180 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd162317-3840-4773-955e-4b1b93fd0e0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=250a212af3c9448b84bb5b049b604942)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:15.554457 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd162317-3840-4773-955e-4b1b93fd0e0f admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:15.604813 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-dd162317-3840-4773-955e-4b1b93fd0e0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '250a212af3c9448b84bb5b049b604942', 'request_id': 'req-dd162317-3840-4773-955e-4b1b93fd0e0f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:15.611749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-dd162317-3840-4773-955e-4b1b93fd0e0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '250a212af3c9448b84bb5b049b604942', 'request_id': 'req-dd162317-3840-4773-955e-4b1b93fd0e0f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:15.620247 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-dd162317-3840-4773-955e-4b1b93fd0e0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '250a212af3c9448b84bb5b049b604942', 'request_id': 'req-dd162317-3840-4773-955e-4b1b93fd0e0f'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:15.632012 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-dd162317-3840-4773-955e-4b1b93fd0e0f admin admin] The token cache is being invalidate because project 250a212af3c9448b84bb5b049b604942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:33:15.632334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-dd162317-3840-4773-955e-4b1b93fd0e0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:15.632975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-dd162317-3840-4773-955e-4b1b93fd0e0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:15.634498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1301/2619] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:33:15 2022] DELETE /identity/v3/projects/250a212af3c9448b84bb5b049b604942 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:15.638505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355042a-96d7-43be-8349-9181bde5a727 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:15.640017 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-f355042a-96d7-43be-8349-9181bde5a727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:15.653427 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355042a-96d7-43be-8349-9181bde5a727 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:15.713104 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc92468-c259-4430-a0d0-814f1c6a512a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:15.713777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc92468-c259-4430-a0d0-814f1c6a512a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:15.714055 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc92468-c259-4430-a0d0-814f1c6a512a admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:15.714360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc92468-c259-4430-a0d0-814f1c6a512a admin admin] PATH_INFO: `/v3/users/eedac78a9101497e98e0b3264bfd7055` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:15.715503 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc92468-c259-4430-a0d0-814f1c6a512a admin admin] RBAC: Authorizing `identity:delete_user(user_id=eedac78a9101497e98e0b3264bfd7055)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:15.717433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc92468-c259-4430-a0d0-814f1c6a512a admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:15.733459 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:15.774659 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-abc92468-c259-4430-a0d0-814f1c6a512a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eedac78a9101497e98e0b3264bfd7055'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:15.775350 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-abc92468-c259-4430-a0d0-814f1c6a512a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eedac78a9101497e98e0b3264bfd7055'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:15.786581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-abc92468-c259-4430-a0d0-814f1c6a512a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eedac78a9101497e98e0b3264bfd7055'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:15.787391 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-abc92468-c259-4430-a0d0-814f1c6a512a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eedac78a9101497e98e0b3264bfd7055'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:15.799772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-abc92468-c259-4430-a0d0-814f1c6a512a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eedac78a9101497e98e0b3264bfd7055'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:15.821536 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1319/2620] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:33:15 2022] DELETE /identity/v3/users/eedac78a9101497e98e0b3264bfd7055 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:33:16.134214 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd162317-3840-4773-955e-4b1b93fd0e0f admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:16.135669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-dd162317-3840-4773-955e-4b1b93fd0e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:16.148407 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd162317-3840-4773-955e-4b1b93fd0e0f admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:16.164227 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:16.229110 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2060f-e5d2-43f7-8359-2c62b0653238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:16.229756 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2060f-e5d2-43f7-8359-2c62b0653238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:16.230041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2060f-e5d2-43f7-8359-2c62b0653238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:16.230318 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2060f-e5d2-43f7-8359-2c62b0653238 admin admin] PATH_INFO: `/v3/projects/80e87d5ed2e24caa9571a63981733b46` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:16.231522 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2060f-e5d2-43f7-8359-2c62b0653238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80e87d5ed2e24caa9571a63981733b46)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:16.233455 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2060f-e5d2-43f7-8359-2c62b0653238 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:16.293257 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-cee2060f-e5d2-43f7-8359-2c62b0653238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80e87d5ed2e24caa9571a63981733b46', 'request_id': 'req-cee2060f-e5d2-43f7-8359-2c62b0653238'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:16.299645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-cee2060f-e5d2-43f7-8359-2c62b0653238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80e87d5ed2e24caa9571a63981733b46', 'request_id': 'req-cee2060f-e5d2-43f7-8359-2c62b0653238'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:16.308491 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-cee2060f-e5d2-43f7-8359-2c62b0653238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80e87d5ed2e24caa9571a63981733b46', 'request_id': 'req-cee2060f-e5d2-43f7-8359-2c62b0653238'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:16.321880 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-cee2060f-e5d2-43f7-8359-2c62b0653238 admin admin] The token cache is being invalidate because project 80e87d5ed2e24caa9571a63981733b46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:33:16.321880 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-cee2060f-e5d2-43f7-8359-2c62b0653238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:16.322566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-cee2060f-e5d2-43f7-8359-2c62b0653238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:16.324184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1302/2621] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:33:16 2022] DELETE /identity/v3/projects/80e87d5ed2e24caa9571a63981733b46 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:33:16.329146 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3888f-b92f-4d75-b97c-f6bd2f05f2df None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:16.329441 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3888f-b92f-4d75-b97c-f6bd2f05f2df None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:16.329716 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3888f-b92f-4d75-b97c-f6bd2f05f2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:16.354321 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-74b3888f-b92f-4d75-b97c-f6bd2f05f2df None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:16.355780 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-74b3888f-b92f-4d75-b97c-f6bd2f05f2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:16.426336 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1320/2622] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:16.434837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab48569-7875-4219-8635-a48b08ba1e7e None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:16.435122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab48569-7875-4219-8635-a48b08ba1e7e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:16.435403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab48569-7875-4219-8635-a48b08ba1e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:16.456311 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-1ab48569-7875-4219-8635-a48b08ba1e7e None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:16.457565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1ab48569-7875-4219-8635-a48b08ba1e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:16.465505 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1303/2623] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:33:16.470173 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3888f-b92f-4d75-b97c-f6bd2f05f2df None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:16.480208 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3888f-b92f-4d75-b97c-f6bd2f05f2df None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:16.485964 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cc529-8425-4cec-ac08-bbec3ea8e6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:16.486629 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cc529-8425-4cec-ac08-bbec3ea8e6c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:16.486908 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cc529-8425-4cec-ac08-bbec3ea8e6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:16.487174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cc529-8425-4cec-ac08-bbec3ea8e6c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:16.487812 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637cc529-8425-4cec-ac08-bbec3ea8e6c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:33:16.488112 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637cc529-8425-4cec-ac08-bbec3ea8e6c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:16.489771 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637cc529-8425-4cec-ac08-bbec3ea8e6c9 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:16.502729 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1321/2624] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:33:16.507074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab48569-7875-4219-8635-a48b08ba1e7e None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:16.516491 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab48569-7875-4219-8635-a48b08ba1e7e None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:16.521752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314a8b9-46ae-4627-87df-550981623526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:16.522437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314a8b9-46ae-4627-87df-550981623526 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:16.522805 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314a8b9-46ae-4627-87df-550981623526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:16.523049 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314a8b9-46ae-4627-87df-550981623526 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:16.523846 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314a8b9-46ae-4627-87df-550981623526 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:16.526619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314a8b9-46ae-4627-87df-550981623526 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:16.559274 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1304/2625] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:33:16 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:16.563839 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cc529-8425-4cec-ac08-bbec3ea8e6c9 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:16.574774 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cc529-8425-4cec-ac08-bbec3ea8e6c9 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:16.639814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518bf39-4aa8-40c2-b88a-244a95a5f565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:16.640471 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518bf39-4aa8-40c2-b88a-244a95a5f565 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:16.640616 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518bf39-4aa8-40c2-b88a-244a95a5f565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:16.641062 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518bf39-4aa8-40c2-b88a-244a95a5f565 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:16.641559 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518bf39-4aa8-40c2-b88a-244a95a5f565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:16.643455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518bf39-4aa8-40c2-b88a-244a95a5f565 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:16.675358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1322/2626] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:33:16 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:16.675803 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314a8b9-46ae-4627-87df-550981623526 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:16.692646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314a8b9-46ae-4627-87df-550981623526 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:16.702510 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507fe7f-5d71-4074-9007-b2c697727345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:16.703142 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507fe7f-5d71-4074-9007-b2c697727345 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:16.703421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507fe7f-5d71-4074-9007-b2c697727345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:16.703686 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507fe7f-5d71-4074-9007-b2c697727345 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:16.704330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507fe7f-5d71-4074-9007-b2c697727345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:16.706002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507fe7f-5d71-4074-9007-b2c697727345 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:16.733438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1305/2627] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:16.737414 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518bf39-4aa8-40c2-b88a-244a95a5f565 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:16.754136 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518bf39-4aa8-40c2-b88a-244a95a5f565 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:16.759640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c3200-f23d-4faf-87b2-d665cf93a143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:16.760262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c3200-f23d-4faf-87b2-d665cf93a143 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:16.760544 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c3200-f23d-4faf-87b2-d665cf93a143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:16.760819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c3200-f23d-4faf-87b2-d665cf93a143 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:16.761508 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c3200-f23d-4faf-87b2-d665cf93a143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:16.763262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c3200-f23d-4faf-87b2-d665cf93a143 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:16.765863 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:16.778204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1323/2628] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:16.782496 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507fe7f-5d71-4074-9007-b2c697727345 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:16.798912 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507fe7f-5d71-4074-9007-b2c697727345 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:16.804514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4bfd5-08e1-4e56-8f95-51d64395cbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:16.807350 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4bfd5-08e1-4e56-8f95-51d64395cbbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:16.807650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4bfd5-08e1-4e56-8f95-51d64395cbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:16.807927 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4bfd5-08e1-4e56-8f95-51d64395cbbe admin admin] PATH_INFO: `/v3/projects/36cbeee8a77f4fcfb57944bd6a1a2cda/users/b112be74eea9478b9a5a78359a63c7ec/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:16.825445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4bfd5-08e1-4e56-8f95-51d64395cbbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=36cbeee8a77f4fcfb57944bd6a1a2cda, user_id=b112be74eea9478b9a5a78359a63c7ec, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:16.829008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4bfd5-08e1-4e56-8f95-51d64395cbbe admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:16.848023 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1306/2629] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:16 2022] PUT /identity/v3/projects/36cbeee8a77f4fcfb57944bd6a1a2cda/users/b112be74eea9478b9a5a78359a63c7ec/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:16.858827 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb142a0-894b-4f0b-9a4c-d14b5f7f6e78 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:16.859237 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb142a0-894b-4f0b-9a4c-d14b5f7f6e78 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:16.859592 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb142a0-894b-4f0b-9a4c-d14b5f7f6e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:16.889715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-7eb142a0-894b-4f0b-9a4c-d14b5f7f6e78 None None] MFA Rules not processed for user `b112be74eea9478b9a5a78359a63c7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:16.890971 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7eb142a0-894b-4f0b-9a4c-d14b5f7f6e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:16.981123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1324/2630] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:16.982365 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4bfd5-08e1-4e56-8f95-51d64395cbbe admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:17.024592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:17.048438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4bfd5-08e1-4e56-8f95-51d64395cbbe admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:17.053829 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0ef1b-e196-4f49-a1db-3ee417c76da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:17.054449 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0ef1b-e196-4f49-a1db-3ee417c76da1 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:17.054760 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0ef1b-e196-4f49-a1db-3ee417c76da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:17.055036 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0ef1b-e196-4f49-a1db-3ee417c76da1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:17.055706 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0ef1b-e196-4f49-a1db-3ee417c76da1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:17.057628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0ef1b-e196-4f49-a1db-3ee417c76da1 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:17.088592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1307/2631] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:33:16 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:17.089795 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb142a0-894b-4f0b-9a4c-d14b5f7f6e78 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:17.100627 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb142a0-894b-4f0b-9a4c-d14b5f7f6e78 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:17.196588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e168fb1-b8a4-40d4-a88a-a9ed777213aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:17.197291 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e168fb1-b8a4-40d4-a88a-a9ed777213aa admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:17.197576 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e168fb1-b8a4-40d4-a88a-a9ed777213aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:17.197853 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e168fb1-b8a4-40d4-a88a-a9ed777213aa admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:17.198483 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e168fb1-b8a4-40d4-a88a-a9ed777213aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:17.200387 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e168fb1-b8a4-40d4-a88a-a9ed777213aa admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:17.227535 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1325/2632] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:33:17 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:17.231357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0ef1b-e196-4f49-a1db-3ee417c76da1 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:17.243611 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0ef1b-e196-4f49-a1db-3ee417c76da1 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:17.249026 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b87bc8-9457-4e3d-80f6-ccb4f766bd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:17.249985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b87bc8-9457-4e3d-80f6-ccb4f766bd26 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:17.250369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b87bc8-9457-4e3d-80f6-ccb4f766bd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:17.250705 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b87bc8-9457-4e3d-80f6-ccb4f766bd26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:17.251425 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b87bc8-9457-4e3d-80f6-ccb4f766bd26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:17.253039 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b87bc8-9457-4e3d-80f6-ccb4f766bd26 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:17.267860 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1308/2633] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:17.269153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e168fb1-b8a4-40d4-a88a-a9ed777213aa admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:17.281046 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e168fb1-b8a4-40d4-a88a-a9ed777213aa admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:17.291620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9915bc-ff14-47ff-b4f8-ec8c4657cbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:17.292341 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9915bc-ff14-47ff-b4f8-ec8c4657cbaf admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:17.292688 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9915bc-ff14-47ff-b4f8-ec8c4657cbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:17.293167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9915bc-ff14-47ff-b4f8-ec8c4657cbaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:17.293835 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9915bc-ff14-47ff-b4f8-ec8c4657cbaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:17.295431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9915bc-ff14-47ff-b4f8-ec8c4657cbaf admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:17.307129 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1326/2634] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:17.311442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b87bc8-9457-4e3d-80f6-ccb4f766bd26 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:17.322866 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b87bc8-9457-4e3d-80f6-ccb4f766bd26 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:17.336487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cbe04c-81d4-455d-9616-d23c5a9776ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:17.337204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cbe04c-81d4-455d-9616-d23c5a9776ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:17.337492 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cbe04c-81d4-455d-9616-d23c5a9776ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:17.337780 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cbe04c-81d4-455d-9616-d23c5a9776ec admin admin] PATH_INFO: `/v3/projects/c3ff7c2a2f8e48b29121edb9cf13087b/users/9a3095ccf7d1415e888ae3bb66c5621c/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:17.356277 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cbe04c-81d4-455d-9616-d23c5a9776ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3ff7c2a2f8e48b29121edb9cf13087b, user_id=9a3095ccf7d1415e888ae3bb66c5621c, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:17.359025 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cbe04c-81d4-455d-9616-d23c5a9776ec admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:17.376443 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1309/2635] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:17 2022] PUT /identity/v3/projects/c3ff7c2a2f8e48b29121edb9cf13087b/users/9a3095ccf7d1415e888ae3bb66c5621c/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:17.446325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8411bed-e36a-49c2-85de-7ff498ea4294 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:17.446654 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8411bed-e36a-49c2-85de-7ff498ea4294 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:17.446888 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8411bed-e36a-49c2-85de-7ff498ea4294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:17.477330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-b8411bed-e36a-49c2-85de-7ff498ea4294 None None] MFA Rules not processed for user `9a3095ccf7d1415e888ae3bb66c5621c`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:17.478541 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b8411bed-e36a-49c2-85de-7ff498ea4294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:17.535361 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:17.546624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1327/2636] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:17.550841 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cbe04c-81d4-455d-9616-d23c5a9776ec admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:17.617731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cbe04c-81d4-455d-9616-d23c5a9776ec admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:17.622457 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2739b1-a0d1-4f81-89f6-299fca308e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:17.623040 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2739b1-a0d1-4f81-89f6-299fca308e29 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:17.623317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2739b1-a0d1-4f81-89f6-299fca308e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:17.623609 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2739b1-a0d1-4f81-89f6-299fca308e29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:17.624223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2739b1-a0d1-4f81-89f6-299fca308e29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:17.626162 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2739b1-a0d1-4f81-89f6-299fca308e29 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:17.659211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1310/2637] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:33:17 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:33:17.660394 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8411bed-e36a-49c2-85de-7ff498ea4294 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:17.666650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2739b1-a0d1-4f81-89f6-299fca308e29 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:17.668358 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-df2739b1-a0d1-4f81-89f6-299fca308e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:17.676570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8411bed-e36a-49c2-85de-7ff498ea4294 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:17.732367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2739b1-a0d1-4f81-89f6-299fca308e29 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:17.735299 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478297d7-4f83-4d54-b1ed-d47bc516fc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:17.735908 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478297d7-4f83-4d54-b1ed-d47bc516fc66 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:17.736184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478297d7-4f83-4d54-b1ed-d47bc516fc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:17.736449 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478297d7-4f83-4d54-b1ed-d47bc516fc66 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:17.737065 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478297d7-4f83-4d54-b1ed-d47bc516fc66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:17.737359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03404d20-b4a7-40dc-8ead-6a3ef593cd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:17.737992 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03404d20-b4a7-40dc-8ead-6a3ef593cd37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:17.738272 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03404d20-b4a7-40dc-8ead-6a3ef593cd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:17.738551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03404d20-b4a7-40dc-8ead-6a3ef593cd37 admin admin] PATH_INFO: `/v3/users/4d7b7c68a699443aa5c0859094c265ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:17.739003 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478297d7-4f83-4d54-b1ed-d47bc516fc66 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:17.753318 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03404d20-b4a7-40dc-8ead-6a3ef593cd37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d7b7c68a699443aa5c0859094c265ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:17.755243 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03404d20-b4a7-40dc-8ead-6a3ef593cd37 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:17.758395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:17.762538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1328/2638] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:33:17 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:17.766447 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478297d7-4f83-4d54-b1ed-d47bc516fc66 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:17.776070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478297d7-4f83-4d54-b1ed-d47bc516fc66 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:17.781044 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337acc6-4988-495f-9933-53ac2f47f965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:17.781685 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337acc6-4988-495f-9933-53ac2f47f965 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:17.781965 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337acc6-4988-495f-9933-53ac2f47f965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:17.782232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337acc6-4988-495f-9933-53ac2f47f965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:17.783009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337acc6-4988-495f-9933-53ac2f47f965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:17.784829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337acc6-4988-495f-9933-53ac2f47f965 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:17.796163 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1329/2639] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:17.800624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337acc6-4988-495f-9933-53ac2f47f965 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:17.810556 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-03404d20-b4a7-40dc-8ead-6a3ef593cd37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d7b7c68a699443aa5c0859094c265ec'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:17.811014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337acc6-4988-495f-9933-53ac2f47f965 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:17.811230 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-03404d20-b4a7-40dc-8ead-6a3ef593cd37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d7b7c68a699443aa5c0859094c265ec'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:17.815696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed759a62-dd7e-4b0b-8176-1e3ba835a155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:17.816243 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed759a62-dd7e-4b0b-8176-1e3ba835a155 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:17.816525 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed759a62-dd7e-4b0b-8176-1e3ba835a155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:17.816799 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed759a62-dd7e-4b0b-8176-1e3ba835a155 admin admin] PATH_INFO: `/v3/projects/62decd0b2024443bb54e3487bf451551/users/07ce60b581fd449ab91eab3e80d3838c/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:17.824277 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-03404d20-b4a7-40dc-8ead-6a3ef593cd37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d7b7c68a699443aa5c0859094c265ec'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:17.832042 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-03404d20-b4a7-40dc-8ead-6a3ef593cd37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d7b7c68a699443aa5c0859094c265ec'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:17.832448 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed759a62-dd7e-4b0b-8176-1e3ba835a155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62decd0b2024443bb54e3487bf451551, user_id=07ce60b581fd449ab91eab3e80d3838c, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:17.832983 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-03404d20-b4a7-40dc-8ead-6a3ef593cd37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d7b7c68a699443aa5c0859094c265ec'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:17.835793 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed759a62-dd7e-4b0b-8176-1e3ba835a155 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:17.847757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1311/2640] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:33:17 2022] DELETE /identity/v3/users/4d7b7c68a699443aa5c0859094c265ec => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:33:17.850622 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1330/2641] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:17 2022] PUT /identity/v3/projects/62decd0b2024443bb54e3487bf451551/users/07ce60b581fd449ab91eab3e80d3838c/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:17.854452 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03404d20-b4a7-40dc-8ead-6a3ef593cd37 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:17.855844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-03404d20-b4a7-40dc-8ead-6a3ef593cd37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:17.929979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03404d20-b4a7-40dc-8ead-6a3ef593cd37 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:17.934787 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd53283-1aba-4050-b486-b9279444bfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:17.935401 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd53283-1aba-4050-b486-b9279444bfd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:17.935685 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd53283-1aba-4050-b486-b9279444bfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:17.935959 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd53283-1aba-4050-b486-b9279444bfd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:17.936600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd53283-1aba-4050-b486-b9279444bfd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:17.938225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd53283-1aba-4050-b486-b9279444bfd0 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:17.949005 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1312/2642] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:17.953602 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed759a62-dd7e-4b0b-8176-1e3ba835a155 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:18.018585 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed759a62-dd7e-4b0b-8176-1e3ba835a155 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:18.023700 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87cbc2-d83c-41a4-9e0c-255fb32ef5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:18.024372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87cbc2-d83c-41a4-9e0c-255fb32ef5d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:18.024704 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87cbc2-d83c-41a4-9e0c-255fb32ef5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:18.024933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87cbc2-d83c-41a4-9e0c-255fb32ef5d4 admin admin] PATH_INFO: `/v3/projects/62decd0b2024443bb54e3487bf451551/users/07ce60b581fd449ab91eab3e80d3838c/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:18.026996 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87cbc2-d83c-41a4-9e0c-255fb32ef5d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62decd0b2024443bb54e3487bf451551, user_id=07ce60b581fd449ab91eab3e80d3838c, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:18.029623 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87cbc2-d83c-41a4-9e0c-255fb32ef5d4 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:18.035138 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd53283-1aba-4050-b486-b9279444bfd0 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:18.037388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5dd53283-1aba-4050-b486-b9279444bfd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:18.045221 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1331/2643] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:17 2022] PUT /identity/v3/projects/62decd0b2024443bb54e3487bf451551/users/07ce60b581fd449ab91eab3e80d3838c/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:18.052290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c73d8-1fa3-45ea-9aef-9a992dcbb536 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:18.053174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c73d8-1fa3-45ea-9aef-9a992dcbb536 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:18.053515 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c73d8-1fa3-45ea-9aef-9a992dcbb536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:18.053694 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd53283-1aba-4050-b486-b9279444bfd0 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:18.082169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-377c73d8-1fa3-45ea-9aef-9a992dcbb536 None None] MFA Rules not processed for user `07ce60b581fd449ab91eab3e80d3838c`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:18.083474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-377c73d8-1fa3-45ea-9aef-9a992dcbb536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:18.112849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc6a81-5c70-4502-a0aa-099274ac87b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:18.113511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc6a81-5c70-4502-a0aa-099274ac87b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:18.113794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc6a81-5c70-4502-a0aa-099274ac87b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:18.114066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc6a81-5c70-4502-a0aa-099274ac87b9 admin admin] PATH_INFO: `/v3/projects/a396283cd8314a74bfd5e411c246cc96` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:18.119989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:18.126507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc6a81-5c70-4502-a0aa-099274ac87b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a396283cd8314a74bfd5e411c246cc96)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:18.128453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc6a81-5c70-4502-a0aa-099274ac87b9 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:18.139352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:18.157937 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1332/2644] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:18.171182 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c73d8-1fa3-45ea-9aef-9a992dcbb536 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:18.172577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-377c73d8-1fa3-45ea-9aef-9a992dcbb536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:18.181944 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-16fc6a81-5c70-4502-a0aa-099274ac87b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a396283cd8314a74bfd5e411c246cc96', 'request_id': 'req-16fc6a81-5c70-4502-a0aa-099274ac87b9'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:18.191732 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-16fc6a81-5c70-4502-a0aa-099274ac87b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a396283cd8314a74bfd5e411c246cc96', 'request_id': 'req-16fc6a81-5c70-4502-a0aa-099274ac87b9'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:18.200298 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-16fc6a81-5c70-4502-a0aa-099274ac87b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a396283cd8314a74bfd5e411c246cc96', 'request_id': 'req-16fc6a81-5c70-4502-a0aa-099274ac87b9'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:18.211009 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-16fc6a81-5c70-4502-a0aa-099274ac87b9 admin admin] The token cache is being invalidate because project a396283cd8314a74bfd5e411c246cc96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:33:18.211304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-16fc6a81-5c70-4502-a0aa-099274ac87b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:18.211960 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-16fc6a81-5c70-4502-a0aa-099274ac87b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:18.213521 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1313/2645] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:33:18 2022] DELETE /identity/v3/projects/a396283cd8314a74bfd5e411c246cc96 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:33:18.218543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34be7ed-8294-4fc9-a555-69178787f6a7 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:18.218830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34be7ed-8294-4fc9-a555-69178787f6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:18.219103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34be7ed-8294-4fc9-a555-69178787f6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:18.237119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c73d8-1fa3-45ea-9aef-9a992dcbb536 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:18.238507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-c34be7ed-8294-4fc9-a555-69178787f6a7 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:18.239709 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c34be7ed-8294-4fc9-a555-69178787f6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:18.287085 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bc653-e93c-470a-ab02-1ae63e90be55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:18.287668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bc653-e93c-470a-ab02-1ae63e90be55 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:18.287947 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bc653-e93c-470a-ab02-1ae63e90be55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:18.288217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567bc653-e93c-470a-ab02-1ae63e90be55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:18.290057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-567bc653-e93c-470a-ab02-1ae63e90be55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:18.300718 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1314/2646] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:33:18.308861 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531e0c8-fa24-4939-9f07-61008fcd98c9 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:18.309159 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531e0c8-fa24-4939-9f07-61008fcd98c9 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:18.309438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531e0c8-fa24-4939-9f07-61008fcd98c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:18.330423 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-4531e0c8-fa24-4939-9f07-61008fcd98c9 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:18.331635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-4531e0c8-fa24-4939-9f07-61008fcd98c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:18.339996 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1315/2647] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:33:18.344165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531e0c8-fa24-4939-9f07-61008fcd98c9 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:18.353453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531e0c8-fa24-4939-9f07-61008fcd98c9 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:18.358028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efe611-282d-4ce3-9d66-c8faa9842d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:18.358615 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efe611-282d-4ce3-9d66-c8faa9842d93 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:18.358891 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efe611-282d-4ce3-9d66-c8faa9842d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:18.359164 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efe611-282d-4ce3-9d66-c8faa9842d93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:18.359788 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6efe611-282d-4ce3-9d66-c8faa9842d93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:33:18.360087 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6efe611-282d-4ce3-9d66-c8faa9842d93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:18.361705 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6efe611-282d-4ce3-9d66-c8faa9842d93 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:18.364707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bc653-e93c-470a-ab02-1ae63e90be55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:18.366897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567bc653-e93c-470a-ab02-1ae63e90be55 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:18.374144 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1316/2648] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:33:18.375993 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1333/2649] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:33:18.378299 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efe611-282d-4ce3-9d66-c8faa9842d93 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:18.389537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efe611-282d-4ce3-9d66-c8faa9842d93 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:18.394780 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c13743-5df8-4025-8800-55c2fbede5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:18.395533 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c13743-5df8-4025-8800-55c2fbede5cc admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:18.395826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c13743-5df8-4025-8800-55c2fbede5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:18.396101 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c13743-5df8-4025-8800-55c2fbede5cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:18.396767 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c13743-5df8-4025-8800-55c2fbede5cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:18.401330 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c13743-5df8-4025-8800-55c2fbede5cc admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:18.435817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1317/2650] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:33:18 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:18.439805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bc653-e93c-470a-ab02-1ae63e90be55 service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:18.451279 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567bc653-e93c-470a-ab02-1ae63e90be55 service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:18.492625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:18.513813 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4dc69-f4e8-49ed-8205-29b21445151c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:18.514434 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4dc69-f4e8-49ed-8205-29b21445151c admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:18.514766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4dc69-f4e8-49ed-8205-29b21445151c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:18.515063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4dc69-f4e8-49ed-8205-29b21445151c admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:18.515749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4dc69-f4e8-49ed-8205-29b21445151c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:18.518001 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4dc69-f4e8-49ed-8205-29b21445151c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:18.540164 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1334/2651] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:33:18 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:33:18.544639 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c13743-5df8-4025-8800-55c2fbede5cc admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:18.555767 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c13743-5df8-4025-8800-55c2fbede5cc admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:18.560672 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a282b-c1e7-4510-941a-a24f4cd2f252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:18.561313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a282b-c1e7-4510-941a-a24f4cd2f252 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:18.561592 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a282b-c1e7-4510-941a-a24f4cd2f252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:18.561860 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a282b-c1e7-4510-941a-a24f4cd2f252 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:18.562494 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469a282b-c1e7-4510-941a-a24f4cd2f252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:18.564102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469a282b-c1e7-4510-941a-a24f4cd2f252 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:18.577956 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1318/2652] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:18.579290 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4dc69-f4e8-49ed-8205-29b21445151c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:18.589867 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4dc69-f4e8-49ed-8205-29b21445151c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:18.594680 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c8757-1ad9-4440-9ed1-bc732bdc5313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:18.595274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c8757-1ad9-4440-9ed1-bc732bdc5313 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:18.595551 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c8757-1ad9-4440-9ed1-bc732bdc5313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:18.595819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c8757-1ad9-4440-9ed1-bc732bdc5313 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:18.596497 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c8757-1ad9-4440-9ed1-bc732bdc5313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:18.598102 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c8757-1ad9-4440-9ed1-bc732bdc5313 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:18.608890 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1335/2653] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:18.614074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a282b-c1e7-4510-941a-a24f4cd2f252 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:18.625276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469a282b-c1e7-4510-941a-a24f4cd2f252 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:18.634645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460588ac-f963-4683-819b-a6e6ccd8e144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:18.635519 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460588ac-f963-4683-819b-a6e6ccd8e144 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:18.635862 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460588ac-f963-4683-819b-a6e6ccd8e144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:18.636203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460588ac-f963-4683-819b-a6e6ccd8e144 admin admin] PATH_INFO: `/v3/projects/4f67b5c236854a2b84641da5360d394d/users/40804320fbb447439af2738d334b4c43/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:18.652528 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460588ac-f963-4683-819b-a6e6ccd8e144 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f67b5c236854a2b84641da5360d394d, user_id=40804320fbb447439af2738d334b4c43, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:18.655305 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460588ac-f963-4683-819b-a6e6ccd8e144 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:18.668636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1319/2654] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:18 2022] PUT /identity/v3/projects/4f67b5c236854a2b84641da5360d394d/users/40804320fbb447439af2738d334b4c43/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:18.680704 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef069fa6-71ad-43b6-85a7-3c23d51183d3 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:18.681228 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef069fa6-71ad-43b6-85a7-3c23d51183d3 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:18.681468 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef069fa6-71ad-43b6-85a7-3c23d51183d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:18.715253 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-ef069fa6-71ad-43b6-85a7-3c23d51183d3 None None] MFA Rules not processed for user `40804320fbb447439af2738d334b4c43`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:18.715253 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-ef069fa6-71ad-43b6-85a7-3c23d51183d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:18.777289 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1336/2655] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:18.781812 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460588ac-f963-4683-819b-a6e6ccd8e144 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:18.806824 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:18.845487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460588ac-f963-4683-819b-a6e6ccd8e144 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:18.850288 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5779019f-308a-4a31-b8e4-a09658b7f0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:18.850885 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779019f-308a-4a31-b8e4-a09658b7f0e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:18.851167 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779019f-308a-4a31-b8e4-a09658b7f0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:18.851442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779019f-308a-4a31-b8e4-a09658b7f0e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:18.852061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5779019f-308a-4a31-b8e4-a09658b7f0e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:18.853978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5779019f-308a-4a31-b8e4-a09658b7f0e6 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:18.879545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1320/2656] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:33:18 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:18.884268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef069fa6-71ad-43b6-85a7-3c23d51183d3 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:18.895379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef069fa6-71ad-43b6-85a7-3c23d51183d3 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:18.950287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cffdc-88f2-44db-b96e-8b6487abe0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:18.950796 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cffdc-88f2-44db-b96e-8b6487abe0ff admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:18.951076 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cffdc-88f2-44db-b96e-8b6487abe0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:18.951350 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cffdc-88f2-44db-b96e-8b6487abe0ff admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:18.951969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cffdc-88f2-44db-b96e-8b6487abe0ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:18.953882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cffdc-88f2-44db-b96e-8b6487abe0ff admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:18.975236 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1337/2657] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:33:18 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:18.979190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5779019f-308a-4a31-b8e4-a09658b7f0e6 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:18.989393 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5779019f-308a-4a31-b8e4-a09658b7f0e6 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:18.994235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a678e-bca2-49ac-9f80-a5a1dec9ab42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:18.995102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a678e-bca2-49ac-9f80-a5a1dec9ab42 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:18.995384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a678e-bca2-49ac-9f80-a5a1dec9ab42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:18.995654 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a678e-bca2-49ac-9f80-a5a1dec9ab42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:18.996287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a678e-bca2-49ac-9f80-a5a1dec9ab42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:18.997890 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a678e-bca2-49ac-9f80-a5a1dec9ab42 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:19.008793 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1321/2658] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:19.012929 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cffdc-88f2-44db-b96e-8b6487abe0ff admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:19.023317 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cffdc-88f2-44db-b96e-8b6487abe0ff admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:19.027867 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d73174-e1d7-43ab-bbe3-3a7b40d943ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:19.028507 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d73174-e1d7-43ab-bbe3-3a7b40d943ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:19.028788 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d73174-e1d7-43ab-bbe3-3a7b40d943ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:19.029061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d73174-e1d7-43ab-bbe3-3a7b40d943ab admin admin] PATH_INFO: `/v3/projects/6a3b336583954ea696228cb5fad979ac/users/746d45fea01a40c58c23c93db8b6e40c/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:19.043257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d73174-e1d7-43ab-bbe3-3a7b40d943ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a3b336583954ea696228cb5fad979ac, user_id=746d45fea01a40c58c23c93db8b6e40c, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:19.045933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d73174-e1d7-43ab-bbe3-3a7b40d943ab admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:19.059100 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1338/2659] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:19 2022] PUT /identity/v3/projects/6a3b336583954ea696228cb5fad979ac/users/746d45fea01a40c58c23c93db8b6e40c/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:33:19.062959 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a678e-bca2-49ac-9f80-a5a1dec9ab42 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:19.073355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a678e-bca2-49ac-9f80-a5a1dec9ab42 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:19.128903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b36dc-bd99-43f4-b719-6c260c5dad97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:19.129532 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b36dc-bd99-43f4-b719-6c260c5dad97 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:19.129809 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b36dc-bd99-43f4-b719-6c260c5dad97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:19.130079 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b36dc-bd99-43f4-b719-6c260c5dad97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:19.130818 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b36dc-bd99-43f4-b719-6c260c5dad97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:19.132405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b36dc-bd99-43f4-b719-6c260c5dad97 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:19.141883 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1322/2660] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:19.145934 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d73174-e1d7-43ab-bbe3-3a7b40d943ab admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:19.156330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d73174-e1d7-43ab-bbe3-3a7b40d943ab admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:19.160789 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac8db4-4550-4349-a379-dd7150a75c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:19.161459 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac8db4-4550-4349-a379-dd7150a75c0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:19.161740 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac8db4-4550-4349-a379-dd7150a75c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:19.162590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac8db4-4550-4349-a379-dd7150a75c0e admin admin] PATH_INFO: `/v3/projects/6a3b336583954ea696228cb5fad979ac/users/746d45fea01a40c58c23c93db8b6e40c/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:19.164548 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac8db4-4550-4349-a379-dd7150a75c0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a3b336583954ea696228cb5fad979ac, user_id=746d45fea01a40c58c23c93db8b6e40c, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:19.167243 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac8db4-4550-4349-a379-dd7150a75c0e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:19.173814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:19.178867 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1339/2661] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:19 2022] PUT /identity/v3/projects/6a3b336583954ea696228cb5fad979ac/users/746d45fea01a40c58c23c93db8b6e40c/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:33:19.189465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a13bf-7c05-4b9c-8dc3-09c770be14dc None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:19.189779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a13bf-7c05-4b9c-8dc3-09c770be14dc None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:19.190073 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a13bf-7c05-4b9c-8dc3-09c770be14dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:19.218865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-7e7a13bf-7c05-4b9c-8dc3-09c770be14dc None None] MFA Rules not processed for user `746d45fea01a40c58c23c93db8b6e40c`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:19.220070 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7e7a13bf-7c05-4b9c-8dc3-09c770be14dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:19.289778 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1323/2662] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:19.301574 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac8db4-4550-4349-a379-dd7150a75c0e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:19.303772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-7fac8db4-4550-4349-a379-dd7150a75c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:19.364166 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac8db4-4550-4349-a379-dd7150a75c0e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:19.368940 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f0328-bab7-4209-b04b-94f6e3f94e22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:19.369580 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f0328-bab7-4209-b04b-94f6e3f94e22 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:19.369862 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f0328-bab7-4209-b04b-94f6e3f94e22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:19.370135 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f0328-bab7-4209-b04b-94f6e3f94e22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:19.379006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f0328-bab7-4209-b04b-94f6e3f94e22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:19.380405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f0328-bab7-4209-b04b-94f6e3f94e22 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:19.390080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1340/2663] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:19.770841 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a13bf-7c05-4b9c-8dc3-09c770be14dc None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:19.791281 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:19.838705 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a13bf-7c05-4b9c-8dc3-09c770be14dc None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:19.843569 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da2392-cc43-4852-93e2-9ae674ba8c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:19.844211 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da2392-cc43-4852-93e2-9ae674ba8c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:19.844490 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da2392-cc43-4852-93e2-9ae674ba8c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:19.844766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da2392-cc43-4852-93e2-9ae674ba8c6d admin admin] PATH_INFO: `/v3/users/40804320fbb447439af2738d334b4c43` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:19.845938 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da2392-cc43-4852-93e2-9ae674ba8c6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=40804320fbb447439af2738d334b4c43)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:19.847854 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da2392-cc43-4852-93e2-9ae674ba8c6d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:19.906747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-07da2392-cc43-4852-93e2-9ae674ba8c6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40804320fbb447439af2738d334b4c43'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:19.907413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-07da2392-cc43-4852-93e2-9ae674ba8c6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40804320fbb447439af2738d334b4c43'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:19.919969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-07da2392-cc43-4852-93e2-9ae674ba8c6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40804320fbb447439af2738d334b4c43'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:19.929492 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-07da2392-cc43-4852-93e2-9ae674ba8c6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40804320fbb447439af2738d334b4c43'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:19.930149 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-07da2392-cc43-4852-93e2-9ae674ba8c6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40804320fbb447439af2738d334b4c43'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:19.934775 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f0328-bab7-4209-b04b-94f6e3f94e22 service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:19.936263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-ce4f0328-bab7-4209-b04b-94f6e3f94e22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:19.943957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1324/2664] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:33:19 2022] DELETE /identity/v3/users/40804320fbb447439af2738d334b4c43 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:19.948267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88b513-4976-47e7-aa73-c345a8d9b0da None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:19.948603 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88b513-4976-47e7-aa73-c345a8d9b0da None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:19.948956 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f0328-bab7-4209-b04b-94f6e3f94e22 service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:19.949081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88b513-4976-47e7-aa73-c345a8d9b0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:19.968386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-db88b513-4976-47e7-aa73-c345a8d9b0da None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:19.969625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-db88b513-4976-47e7-aa73-c345a8d9b0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:20.001295 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9dc48-b5f2-45e1-a456-6cc98847b4fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:20.001907 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9dc48-b5f2-45e1-a456-6cc98847b4fb service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:20.002178 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9dc48-b5f2-45e1-a456-6cc98847b4fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:20.002446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9dc48-b5f2-45e1-a456-6cc98847b4fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:20.004156 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-45c9dc48-b5f2-45e1-a456-6cc98847b4fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:20.030177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1325/2665] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:33:20.039846 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88b513-4976-47e7-aa73-c345a8d9b0da None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:20.041239 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-db88b513-4976-47e7-aa73-c345a8d9b0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:20.068657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9dc48-b5f2-45e1-a456-6cc98847b4fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:20.070145 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9dc48-b5f2-45e1-a456-6cc98847b4fb service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:20.079070 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1341/2666] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:33:20.083467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:20.100668 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88b513-4976-47e7-aa73-c345a8d9b0da None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:20.104331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1cec6-2d37-48ed-aa58-65a639b14a61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:20.104936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1cec6-2d37-48ed-aa58-65a639b14a61 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:20.105247 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1cec6-2d37-48ed-aa58-65a639b14a61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:20.105524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1cec6-2d37-48ed-aa58-65a639b14a61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:20.114749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d1cec6-2d37-48ed-aa58-65a639b14a61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:20.116716 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d1cec6-2d37-48ed-aa58-65a639b14a61 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:20.126177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1326/2667] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:33:20.407042 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9dc48-b5f2-45e1-a456-6cc98847b4fb service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:20.408719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-45c9dc48-b5f2-45e1-a456-6cc98847b4fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:20.424491 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9dc48-b5f2-45e1-a456-6cc98847b4fb service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:20.429747 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:20.430379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:20.430773 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:20.431088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf admin admin] PATH_INFO: `/v3/projects/4f67b5c236854a2b84641da5360d394d` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:20.432176 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f67b5c236854a2b84641da5360d394d)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:20.434157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:20.485450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f67b5c236854a2b84641da5360d394d', 'request_id': 'req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:20.487455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:20.492638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f67b5c236854a2b84641da5360d394d', 'request_id': 'req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:20.500000 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f67b5c236854a2b84641da5360d394d', 'request_id': 'req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:20.514539 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf admin admin] The token cache is being invalidate because project 4f67b5c236854a2b84641da5360d394d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:33:20.514831 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:20.515479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:20.520215 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1342/2668] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:33:20 2022] DELETE /identity/v3/projects/4f67b5c236854a2b84641da5360d394d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:20.521538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1cec6-2d37-48ed-aa58-65a639b14a61 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:20.530955 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1cec6-2d37-48ed-aa58-65a639b14a61 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:20.591773 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571dd6c-7efd-493b-b7ae-31399b0ce645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:20.592411 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571dd6c-7efd-493b-b7ae-31399b0ce645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:20.593063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571dd6c-7efd-493b-b7ae-31399b0ce645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:20.593372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571dd6c-7efd-493b-b7ae-31399b0ce645 admin admin] PATH_INFO: `/v3/users/746d45fea01a40c58c23c93db8b6e40c` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:20.594523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571dd6c-7efd-493b-b7ae-31399b0ce645 admin admin] RBAC: Authorizing `identity:delete_user(user_id=746d45fea01a40c58c23c93db8b6e40c)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:20.596424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571dd6c-7efd-493b-b7ae-31399b0ce645 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:20.644552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-0571dd6c-7efd-493b-b7ae-31399b0ce645 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '746d45fea01a40c58c23c93db8b6e40c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:20.645227 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-0571dd6c-7efd-493b-b7ae-31399b0ce645 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '746d45fea01a40c58c23c93db8b6e40c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:20.657473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:20.658840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:20.661809 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-0571dd6c-7efd-493b-b7ae-31399b0ce645 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '746d45fea01a40c58c23c93db8b6e40c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:20.667965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-0571dd6c-7efd-493b-b7ae-31399b0ce645 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '746d45fea01a40c58c23c93db8b6e40c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:20.668594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-0571dd6c-7efd-493b-b7ae-31399b0ce645 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '746d45fea01a40c58c23c93db8b6e40c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:20.681783 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1327/2669] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:33:20 2022] DELETE /identity/v3/users/746d45fea01a40c58c23c93db8b6e40c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:33:20.722945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbfea9-8cbb-46c9-b9f2-a0caaea88bdf admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:20.780661 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e66e42-b4be-406f-bba4-dcfc77a62699 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:20.781504 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e66e42-b4be-406f-bba4-dcfc77a62699 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:20.781848 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e66e42-b4be-406f-bba4-dcfc77a62699 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:20.782243 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e66e42-b4be-406f-bba4-dcfc77a62699 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:20.784328 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-71e66e42-b4be-406f-bba4-dcfc77a62699 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:20.850830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e66e42-b4be-406f-bba4-dcfc77a62699 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:20.852267 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e66e42-b4be-406f-bba4-dcfc77a62699 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:20.862257 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1343/2670] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:33:20.970267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571dd6c-7efd-493b-b7ae-31399b0ce645 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:20.981413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0571dd6c-7efd-493b-b7ae-31399b0ce645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:21.038077 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571dd6c-7efd-493b-b7ae-31399b0ce645 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:21.042675 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd395963-21c1-489a-a0f4-92428d88fd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:21.043304 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd395963-21c1-489a-a0f4-92428d88fd86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:21.043583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd395963-21c1-489a-a0f4-92428d88fd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:21.043856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd395963-21c1-489a-a0f4-92428d88fd86 admin admin] PATH_INFO: `/v3/projects/6a3b336583954ea696228cb5fad979ac` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:21.044960 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd395963-21c1-489a-a0f4-92428d88fd86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a3b336583954ea696228cb5fad979ac)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:21.046887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd395963-21c1-489a-a0f4-92428d88fd86 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:21.048299 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:21.094979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-bd395963-21c1-489a-a0f4-92428d88fd86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a3b336583954ea696228cb5fad979ac', 'request_id': 'req-bd395963-21c1-489a-a0f4-92428d88fd86'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:21.101305 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-bd395963-21c1-489a-a0f4-92428d88fd86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a3b336583954ea696228cb5fad979ac', 'request_id': 'req-bd395963-21c1-489a-a0f4-92428d88fd86'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:21.109084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-bd395963-21c1-489a-a0f4-92428d88fd86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a3b336583954ea696228cb5fad979ac', 'request_id': 'req-bd395963-21c1-489a-a0f4-92428d88fd86'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:21.120263 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-bd395963-21c1-489a-a0f4-92428d88fd86 admin admin] The token cache is being invalidate because project 6a3b336583954ea696228cb5fad979ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:33:21.120551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-bd395963-21c1-489a-a0f4-92428d88fd86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:21.121186 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-bd395963-21c1-489a-a0f4-92428d88fd86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:21.122834 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1328/2671] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:33:20 2022] DELETE /identity/v3/projects/6a3b336583954ea696228cb5fad979ac => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:21.127866 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288552a-ece4-4f2e-a0d1-5aae890ab3a3 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:21.128341 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288552a-ece4-4f2e-a0d1-5aae890ab3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:21.128485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288552a-ece4-4f2e-a0d1-5aae890ab3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:21.149317 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-8288552a-ece4-4f2e-a0d1-5aae890ab3a3 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:21.150561 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-8288552a-ece4-4f2e-a0d1-5aae890ab3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:21.194884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:21.214617 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1344/2672] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:33:21.222691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad4e75-b3a8-4cc9-b6db-9bb834d9506a None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:21.222962 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad4e75-b3a8-4cc9-b6db-9bb834d9506a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:21.223236 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad4e75-b3a8-4cc9-b6db-9bb834d9506a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:21.243922 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-18ad4e75-b3a8-4cc9-b6db-9bb834d9506a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:21.245368 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-18ad4e75-b3a8-4cc9-b6db-9bb834d9506a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:21.254037 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1329/2673] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:33:21.258296 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288552a-ece4-4f2e-a0d1-5aae890ab3a3 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:21.268263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288552a-ece4-4f2e-a0d1-5aae890ab3a3 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:21.273715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46246534-4c66-4491-a77d-6813644b285f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:21.274323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46246534-4c66-4491-a77d-6813644b285f admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:21.274600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46246534-4c66-4491-a77d-6813644b285f admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:21.274868 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46246534-4c66-4491-a77d-6813644b285f admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:21.275536 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46246534-4c66-4491-a77d-6813644b285f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:33:21.275833 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46246534-4c66-4491-a77d-6813644b285f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:21.277437 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46246534-4c66-4491-a77d-6813644b285f admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:21.291210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1345/2674] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:33:21.295757 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad4e75-b3a8-4cc9-b6db-9bb834d9506a None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:21.305321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad4e75-b3a8-4cc9-b6db-9bb834d9506a None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:21.309964 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dee90c-8a8c-4f6e-803f-ee0c0d53d233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:21.310560 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dee90c-8a8c-4f6e-803f-ee0c0d53d233 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:21.310837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dee90c-8a8c-4f6e-803f-ee0c0d53d233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:21.311108 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dee90c-8a8c-4f6e-803f-ee0c0d53d233 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:21.311740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dee90c-8a8c-4f6e-803f-ee0c0d53d233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:21.313649 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dee90c-8a8c-4f6e-803f-ee0c0d53d233 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:21.340822 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1330/2675] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:33:21 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:21.345261 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46246534-4c66-4491-a77d-6813644b285f admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:21.355000 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46246534-4c66-4491-a77d-6813644b285f admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:21.410329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6af9a3-b5e8-46a8-88df-adb2f5f0d7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:21.410929 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6af9a3-b5e8-46a8-88df-adb2f5f0d7bc admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:21.411201 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6af9a3-b5e8-46a8-88df-adb2f5f0d7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:21.411479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6af9a3-b5e8-46a8-88df-adb2f5f0d7bc admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:21.412307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6af9a3-b5e8-46a8-88df-adb2f5f0d7bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:21.414309 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6af9a3-b5e8-46a8-88df-adb2f5f0d7bc admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:21.434760 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1346/2676] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:33:21 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:21.438859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dee90c-8a8c-4f6e-803f-ee0c0d53d233 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:21.450064 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dee90c-8a8c-4f6e-803f-ee0c0d53d233 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:21.454782 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eeb340-b810-4345-bda4-a3883058f331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:21.455380 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eeb340-b810-4345-bda4-a3883058f331 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:21.455663 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eeb340-b810-4345-bda4-a3883058f331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:21.455941 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eeb340-b810-4345-bda4-a3883058f331 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:21.456580 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eeb340-b810-4345-bda4-a3883058f331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:21.458219 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eeb340-b810-4345-bda4-a3883058f331 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:21.469358 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1331/2677] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:21.472999 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6af9a3-b5e8-46a8-88df-adb2f5f0d7bc admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:21.482668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6af9a3-b5e8-46a8-88df-adb2f5f0d7bc admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:21.487181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbec627-5c55-4134-b69e-ffcc9d9afd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:21.487761 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbec627-5c55-4134-b69e-ffcc9d9afd58 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:21.488038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbec627-5c55-4134-b69e-ffcc9d9afd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:21.488307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbec627-5c55-4134-b69e-ffcc9d9afd58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:21.488931 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbec627-5c55-4134-b69e-ffcc9d9afd58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:21.490653 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbec627-5c55-4134-b69e-ffcc9d9afd58 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:21.500800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1347/2678] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:21.504707 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eeb340-b810-4345-bda4-a3883058f331 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:21.515361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eeb340-b810-4345-bda4-a3883058f331 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:21.519986 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03f8d8-3b39-405a-9309-a70a584a5e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:21.520628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03f8d8-3b39-405a-9309-a70a584a5e9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:21.520902 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03f8d8-3b39-405a-9309-a70a584a5e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:21.521902 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03f8d8-3b39-405a-9309-a70a584a5e9f admin admin] PATH_INFO: `/v3/projects/40ef41cfe9d042f98b5da706c7743fb0/users/f6141a78337c4ab3b22e3e618449221c/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:21.536074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03f8d8-3b39-405a-9309-a70a584a5e9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ef41cfe9d042f98b5da706c7743fb0, user_id=f6141a78337c4ab3b22e3e618449221c, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:21.538856 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03f8d8-3b39-405a-9309-a70a584a5e9f admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:21.551100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1332/2679] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:21 2022] PUT /identity/v3/projects/40ef41cfe9d042f98b5da706c7743fb0/users/f6141a78337c4ab3b22e3e618449221c/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:21.562431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a60b42-6ca4-406a-83b6-a52bca4a204c None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:21.562711 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a60b42-6ca4-406a-83b6-a52bca4a204c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:21.562981 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a60b42-6ca4-406a-83b6-a52bca4a204c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:21.590236 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-78a60b42-6ca4-406a-83b6-a52bca4a204c None None] MFA Rules not processed for user `f6141a78337c4ab3b22e3e618449221c`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:21.591408 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-78a60b42-6ca4-406a-83b6-a52bca4a204c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:21.655491 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1348/2680] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:33:21.659919 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03f8d8-3b39-405a-9309-a70a584a5e9f admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:21.722371 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03f8d8-3b39-405a-9309-a70a584a5e9f admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:21.727116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281da63e-e132-4465-9e02-3d37d05863fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:21.727716 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281da63e-e132-4465-9e02-3d37d05863fe admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:21.728001 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281da63e-e132-4465-9e02-3d37d05863fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:21.728280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281da63e-e132-4465-9e02-3d37d05863fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:21.728911 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281da63e-e132-4465-9e02-3d37d05863fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:21.730833 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281da63e-e132-4465-9e02-3d37d05863fe admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:21.756982 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1333/2681] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:33:21 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:21.761054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a60b42-6ca4-406a-83b6-a52bca4a204c None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:21.772059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a60b42-6ca4-406a-83b6-a52bca4a204c None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:21.812959 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:21.831573 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f2413-a4af-4b32-8f8e-345d442b42d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:21.832239 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f2413-a4af-4b32-8f8e-345d442b42d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:21.832525 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f2413-a4af-4b32-8f8e-345d442b42d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:21.832798 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f2413-a4af-4b32-8f8e-345d442b42d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:21.833464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f2413-a4af-4b32-8f8e-345d442b42d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:21.835591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f2413-a4af-4b32-8f8e-345d442b42d3 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:21.855657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1349/2682] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:33:21 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:33:21.859871 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281da63e-e132-4465-9e02-3d37d05863fe admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:21.870706 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281da63e-e132-4465-9e02-3d37d05863fe admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:21.875294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a49ec-1f48-4942-a2cf-228e2c43d7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:21.875886 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a49ec-1f48-4942-a2cf-228e2c43d7aa admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:21.876165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a49ec-1f48-4942-a2cf-228e2c43d7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:21.876438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a49ec-1f48-4942-a2cf-228e2c43d7aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:21.877062 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a49ec-1f48-4942-a2cf-228e2c43d7aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:21.878691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a49ec-1f48-4942-a2cf-228e2c43d7aa admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:21.886768 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:21.893188 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1334/2683] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:21.895488 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f2413-a4af-4b32-8f8e-345d442b42d3 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:21.905782 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f2413-a4af-4b32-8f8e-345d442b42d3 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:21.910323 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b21b9-7b00-4067-ad94-b7e455288ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:21.910969 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b21b9-7b00-4067-ad94-b7e455288ca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:21.911246 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b21b9-7b00-4067-ad94-b7e455288ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:21.911517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b21b9-7b00-4067-ad94-b7e455288ca0 admin admin] PATH_INFO: `/v3/projects/b2fac19cacf944909ffeedc038b4a0ab/users/9af29509560347ce938e8a1a10714a49/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:21.925879 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b21b9-7b00-4067-ad94-b7e455288ca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2fac19cacf944909ffeedc038b4a0ab, user_id=9af29509560347ce938e8a1a10714a49, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:21.928987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152b21b9-7b00-4067-ad94-b7e455288ca0 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:21.941577 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1350/2684] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:21 2022] PUT /identity/v3/projects/b2fac19cacf944909ffeedc038b4a0ab/users/9af29509560347ce938e8a1a10714a49/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:21.946099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a49ec-1f48-4942-a2cf-228e2c43d7aa admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:21.955991 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a49ec-1f48-4942-a2cf-228e2c43d7aa admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:22.012749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049a431-8332-46e0-a98b-3e317d871574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:22.013772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049a431-8332-46e0-a98b-3e317d871574 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:22.013772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049a431-8332-46e0-a98b-3e317d871574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:22.013961 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049a431-8332-46e0-a98b-3e317d871574 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:22.014611 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049a431-8332-46e0-a98b-3e317d871574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:22.016215 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049a431-8332-46e0-a98b-3e317d871574 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:22.026343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1335/2685] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:22.030579 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b21b9-7b00-4067-ad94-b7e455288ca0 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:22.040772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b21b9-7b00-4067-ad94-b7e455288ca0 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:22.045401 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16563375-3fa0-4f14-8995-773563ea5d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:22.046041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16563375-3fa0-4f14-8995-773563ea5d97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:22.046316 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16563375-3fa0-4f14-8995-773563ea5d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:22.046583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16563375-3fa0-4f14-8995-773563ea5d97 admin admin] PATH_INFO: `/v3/projects/b2fac19cacf944909ffeedc038b4a0ab/users/9af29509560347ce938e8a1a10714a49/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:22.048604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16563375-3fa0-4f14-8995-773563ea5d97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2fac19cacf944909ffeedc038b4a0ab, user_id=9af29509560347ce938e8a1a10714a49, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:22.051048 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16563375-3fa0-4f14-8995-773563ea5d97 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:22.062236 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1351/2686] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:22 2022] PUT /identity/v3/projects/b2fac19cacf944909ffeedc038b4a0ab/users/9af29509560347ce938e8a1a10714a49/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:22.072813 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc93830-d570-4f3d-b0f1-c752727e593a None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:22.073157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc93830-d570-4f3d-b0f1-c752727e593a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:22.073438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc93830-d570-4f3d-b0f1-c752727e593a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:22.101166 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-cdc93830-d570-4f3d-b0f1-c752727e593a None None] MFA Rules not processed for user `9af29509560347ce938e8a1a10714a49`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:22.102378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-cdc93830-d570-4f3d-b0f1-c752727e593a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:22.176155 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1336/2687] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:22.189414 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16563375-3fa0-4f14-8995-773563ea5d97 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:22.191198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-16563375-3fa0-4f14-8995-773563ea5d97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:22.254757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16563375-3fa0-4f14-8995-773563ea5d97 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:22.258684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabddc7-f5cc-474a-bedb-306d25a29206 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:22.259485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabddc7-f5cc-474a-bedb-306d25a29206 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:22.259636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabddc7-f5cc-474a-bedb-306d25a29206 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:22.259854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabddc7-f5cc-474a-bedb-306d25a29206 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:22.269719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feabddc7-f5cc-474a-bedb-306d25a29206 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:22.271241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feabddc7-f5cc-474a-bedb-306d25a29206 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:22.281583 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1352/2688] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:22.368545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc93830-d570-4f3d-b0f1-c752727e593a None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:22.378430 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc93830-d570-4f3d-b0f1-c752727e593a None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:22.382737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0f5d6-1713-4cef-9564-9ef7ca5926f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:22.383342 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0f5d6-1713-4cef-9564-9ef7ca5926f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:22.383625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0f5d6-1713-4cef-9564-9ef7ca5926f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:22.383899 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0f5d6-1713-4cef-9564-9ef7ca5926f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:22.439894 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0f5d6-1713-4cef-9564-9ef7ca5926f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:22.441357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0f5d6-1713-4cef-9564-9ef7ca5926f3 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:22.450817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1337/2689] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:24.766424 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabddc7-f5cc-474a-bedb-306d25a29206 service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:24.767821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-feabddc7-f5cc-474a-bedb-306d25a29206 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:24.830763 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabddc7-f5cc-474a-bedb-306d25a29206 service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:24.836152 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28991e9-d60d-4a52-aa08-2586db15574c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:24.836786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28991e9-d60d-4a52-aa08-2586db15574c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:24.837060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28991e9-d60d-4a52-aa08-2586db15574c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:24.837359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28991e9-d60d-4a52-aa08-2586db15574c admin admin] PATH_INFO: `/v3/users/b112be74eea9478b9a5a78359a63c7ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:24.838483 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28991e9-d60d-4a52-aa08-2586db15574c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b112be74eea9478b9a5a78359a63c7ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:24.840393 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28991e9-d60d-4a52-aa08-2586db15574c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:24.880892 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:24.888036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c28991e9-d60d-4a52-aa08-2586db15574c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b112be74eea9478b9a5a78359a63c7ec'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:24.888719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c28991e9-d60d-4a52-aa08-2586db15574c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b112be74eea9478b9a5a78359a63c7ec'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:24.896968 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c28991e9-d60d-4a52-aa08-2586db15574c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b112be74eea9478b9a5a78359a63c7ec'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:24.897615 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c28991e9-d60d-4a52-aa08-2586db15574c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b112be74eea9478b9a5a78359a63c7ec'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:24.904517 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c28991e9-d60d-4a52-aa08-2586db15574c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b112be74eea9478b9a5a78359a63c7ec'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:24.922104 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1353/2690] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:33:24 2022] DELETE /identity/v3/users/b112be74eea9478b9a5a78359a63c7ec => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:24.926562 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808ccfd-c62c-4497-a9e4-561dbc17d534 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:24.927062 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808ccfd-c62c-4497-a9e4-561dbc17d534 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:24.927193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808ccfd-c62c-4497-a9e4-561dbc17d534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:24.946733 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-b808ccfd-c62c-4497-a9e4-561dbc17d534 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:24.947937 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b808ccfd-c62c-4497-a9e4-561dbc17d534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:24.961988 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=100 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:25.010302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1338/2691] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:25.020053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28991e9-d60d-4a52-aa08-2586db15574c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:25.022029 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-c28991e9-d60d-4a52-aa08-2586db15574c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:25.078671 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28991e9-d60d-4a52-aa08-2586db15574c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:25.082262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef637205-89a0-48a5-8e10-9d4fb0fb3c64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:25.082861 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef637205-89a0-48a5-8e10-9d4fb0fb3c64 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:25.083678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef637205-89a0-48a5-8e10-9d4fb0fb3c64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:25.083678 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef637205-89a0-48a5-8e10-9d4fb0fb3c64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:25.092368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef637205-89a0-48a5-8e10-9d4fb0fb3c64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:25.094073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef637205-89a0-48a5-8e10-9d4fb0fb3c64 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:25.104472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1354/2692] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:33:25.393537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808ccfd-c62c-4497-a9e4-561dbc17d534 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:25.395377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b808ccfd-c62c-4497-a9e4-561dbc17d534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:25.413429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808ccfd-c62c-4497-a9e4-561dbc17d534 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:25.418028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53842f-c974-40f1-810c-1a5a4d8192e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:25.418666 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53842f-c974-40f1-810c-1a5a4d8192e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:25.418930 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53842f-c974-40f1-810c-1a5a4d8192e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:25.419306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae53842f-c974-40f1-810c-1a5a4d8192e3 admin admin] PATH_INFO: `/v3/projects/36cbeee8a77f4fcfb57944bd6a1a2cda` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:25.420318 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae53842f-c974-40f1-810c-1a5a4d8192e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36cbeee8a77f4fcfb57944bd6a1a2cda)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:25.422253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae53842f-c974-40f1-810c-1a5a4d8192e3 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:25.474495 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ae53842f-c974-40f1-810c-1a5a4d8192e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36cbeee8a77f4fcfb57944bd6a1a2cda', 'request_id': 'req-ae53842f-c974-40f1-810c-1a5a4d8192e3'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:25.480723 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ae53842f-c974-40f1-810c-1a5a4d8192e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36cbeee8a77f4fcfb57944bd6a1a2cda', 'request_id': 'req-ae53842f-c974-40f1-810c-1a5a4d8192e3'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:25.488418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ae53842f-c974-40f1-810c-1a5a4d8192e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36cbeee8a77f4fcfb57944bd6a1a2cda', 'request_id': 'req-ae53842f-c974-40f1-810c-1a5a4d8192e3'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:25.500021 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ae53842f-c974-40f1-810c-1a5a4d8192e3 admin admin] The token cache is being invalidate because project 36cbeee8a77f4fcfb57944bd6a1a2cda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:33:25.500312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ae53842f-c974-40f1-810c-1a5a4d8192e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:25.500949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ae53842f-c974-40f1-810c-1a5a4d8192e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:25.502468 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1339/2693] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:33:25 2022] DELETE /identity/v3/projects/36cbeee8a77f4fcfb57944bd6a1a2cda => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:33:25.506497 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef637205-89a0-48a5-8e10-9d4fb0fb3c64 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:25.516279 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef637205-89a0-48a5-8e10-9d4fb0fb3c64 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:25.574701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f61ff8-3536-4eeb-8f20-32a55757c827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:25.575340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f61ff8-3536-4eeb-8f20-32a55757c827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:25.575628 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f61ff8-3536-4eeb-8f20-32a55757c827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:25.575903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f61ff8-3536-4eeb-8f20-32a55757c827 admin admin] PATH_INFO: `/v3/users/9a3095ccf7d1415e888ae3bb66c5621c` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:25.577021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f61ff8-3536-4eeb-8f20-32a55757c827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a3095ccf7d1415e888ae3bb66c5621c)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:25.578985 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f61ff8-3536-4eeb-8f20-32a55757c827 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:25.627036 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-67f61ff8-3536-4eeb-8f20-32a55757c827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a3095ccf7d1415e888ae3bb66c5621c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:25.627777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-67f61ff8-3536-4eeb-8f20-32a55757c827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a3095ccf7d1415e888ae3bb66c5621c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:25.641430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-67f61ff8-3536-4eeb-8f20-32a55757c827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a3095ccf7d1415e888ae3bb66c5621c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:25.642057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-67f61ff8-3536-4eeb-8f20-32a55757c827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a3095ccf7d1415e888ae3bb66c5621c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:25.647344 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:25.649475 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-67f61ff8-3536-4eeb-8f20-32a55757c827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a3095ccf7d1415e888ae3bb66c5621c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:25.668135 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1355/2694] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:33:25 2022] DELETE /identity/v3/users/9a3095ccf7d1415e888ae3bb66c5621c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:25.936122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53842f-c974-40f1-810c-1a5a4d8192e3 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:25.937683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-ae53842f-c974-40f1-810c-1a5a4d8192e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:25.953032 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae53842f-c974-40f1-810c-1a5a4d8192e3 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:26.011638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0f976-8270-49b1-83f0-f727cd474429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:26.012236 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0f976-8270-49b1-83f0-f727cd474429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:26.012511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0f976-8270-49b1-83f0-f727cd474429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:26.012779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0f976-8270-49b1-83f0-f727cd474429 admin admin] PATH_INFO: `/v3/projects/c3ff7c2a2f8e48b29121edb9cf13087b` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:26.013917 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0f976-8270-49b1-83f0-f727cd474429 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3ff7c2a2f8e48b29121edb9cf13087b)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:26.015819 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0f976-8270-49b1-83f0-f727cd474429 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:26.058247 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:26.061310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ddf0f976-8270-49b1-83f0-f727cd474429 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3ff7c2a2f8e48b29121edb9cf13087b', 'request_id': 'req-ddf0f976-8270-49b1-83f0-f727cd474429'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:26.068952 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ddf0f976-8270-49b1-83f0-f727cd474429 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3ff7c2a2f8e48b29121edb9cf13087b', 'request_id': 'req-ddf0f976-8270-49b1-83f0-f727cd474429'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:26.076734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ddf0f976-8270-49b1-83f0-f727cd474429 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3ff7c2a2f8e48b29121edb9cf13087b', 'request_id': 'req-ddf0f976-8270-49b1-83f0-f727cd474429'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:26.087722 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ddf0f976-8270-49b1-83f0-f727cd474429 admin admin] The token cache is being invalidate because project c3ff7c2a2f8e48b29121edb9cf13087b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:33:26.088017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ddf0f976-8270-49b1-83f0-f727cd474429 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:26.088668 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ddf0f976-8270-49b1-83f0-f727cd474429 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:26.090194 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1340/2695] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:33:25 2022] DELETE /identity/v3/projects/c3ff7c2a2f8e48b29121edb9cf13087b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:26.094407 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f61ff8-3536-4eeb-8f20-32a55757c827 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:26.095857 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-67f61ff8-3536-4eeb-8f20-32a55757c827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:26.108582 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f61ff8-3536-4eeb-8f20-32a55757c827 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:26.163109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb0ed2-f4a8-49e1-add7-42917924e7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:26.163734 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb0ed2-f4a8-49e1-add7-42917924e7bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:26.164014 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb0ed2-f4a8-49e1-add7-42917924e7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:26.164284 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb0ed2-f4a8-49e1-add7-42917924e7bf admin admin] PATH_INFO: `/v3/users/07ce60b581fd449ab91eab3e80d3838c` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:26.165425 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb0ed2-f4a8-49e1-add7-42917924e7bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=07ce60b581fd449ab91eab3e80d3838c)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:26.167326 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb0ed2-f4a8-49e1-add7-42917924e7bf admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:26.218117 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-80eb0ed2-f4a8-49e1-add7-42917924e7bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07ce60b581fd449ab91eab3e80d3838c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:26.218792 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-80eb0ed2-f4a8-49e1-add7-42917924e7bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07ce60b581fd449ab91eab3e80d3838c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:26.226474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-80eb0ed2-f4a8-49e1-add7-42917924e7bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07ce60b581fd449ab91eab3e80d3838c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:26.227196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-80eb0ed2-f4a8-49e1-add7-42917924e7bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07ce60b581fd449ab91eab3e80d3838c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:26.234169 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-80eb0ed2-f4a8-49e1-add7-42917924e7bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07ce60b581fd449ab91eab3e80d3838c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:26.253706 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1356/2696] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:33:26 2022] DELETE /identity/v3/users/07ce60b581fd449ab91eab3e80d3838c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:26.522340 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0f976-8270-49b1-83f0-f727cd474429 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:26.524308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-ddf0f976-8270-49b1-83f0-f727cd474429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:26.538268 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0f976-8270-49b1-83f0-f727cd474429 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:26.597697 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74439d27-f000-4d63-8e53-2068b3be73f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:26.598336 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74439d27-f000-4d63-8e53-2068b3be73f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:26.598619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74439d27-f000-4d63-8e53-2068b3be73f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:26.598895 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74439d27-f000-4d63-8e53-2068b3be73f2 admin admin] PATH_INFO: `/v3/projects/62decd0b2024443bb54e3487bf451551` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:26.600168 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74439d27-f000-4d63-8e53-2068b3be73f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62decd0b2024443bb54e3487bf451551)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:26.602110 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74439d27-f000-4d63-8e53-2068b3be73f2 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:26.649618 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-74439d27-f000-4d63-8e53-2068b3be73f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62decd0b2024443bb54e3487bf451551', 'request_id': 'req-74439d27-f000-4d63-8e53-2068b3be73f2'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:26.656060 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-74439d27-f000-4d63-8e53-2068b3be73f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62decd0b2024443bb54e3487bf451551', 'request_id': 'req-74439d27-f000-4d63-8e53-2068b3be73f2'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:26.663931 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-74439d27-f000-4d63-8e53-2068b3be73f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62decd0b2024443bb54e3487bf451551', 'request_id': 'req-74439d27-f000-4d63-8e53-2068b3be73f2'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:26.678890 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-74439d27-f000-4d63-8e53-2068b3be73f2 admin admin] The token cache is being invalidate because project 62decd0b2024443bb54e3487bf451551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:33:26.679191 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-74439d27-f000-4d63-8e53-2068b3be73f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:26.681553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-74439d27-f000-4d63-8e53-2068b3be73f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:26.683478 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1341/2697] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:33:26 2022] DELETE /identity/v3/projects/62decd0b2024443bb54e3487bf451551 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:26.690636 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825891b-0ddb-4e42-8340-78f760572da7 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:26.690957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825891b-0ddb-4e42-8340-78f760572da7 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:26.691225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825891b-0ddb-4e42-8340-78f760572da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:26.712762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-5825891b-0ddb-4e42-8340-78f760572da7 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:26.713971 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5825891b-0ddb-4e42-8340-78f760572da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:26.763186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:26.773832 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1357/2698] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:26.781549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a5638-3d4d-4b1c-833d-de7de2167808 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:26.781844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a5638-3d4d-4b1c-833d-de7de2167808 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:26.782155 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a5638-3d4d-4b1c-833d-de7de2167808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:26.803720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-1c7a5638-3d4d-4b1c-833d-de7de2167808 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:26.804938 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1c7a5638-3d4d-4b1c-833d-de7de2167808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:26.812940 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1342/2699] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:26.817071 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825891b-0ddb-4e42-8340-78f760572da7 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:26.827292 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825891b-0ddb-4e42-8340-78f760572da7 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:26.831821 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416231ee-ac8e-493b-a35a-1815834d15da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:26.832411 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416231ee-ac8e-493b-a35a-1815834d15da admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:26.832682 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416231ee-ac8e-493b-a35a-1815834d15da admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:26.832946 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416231ee-ac8e-493b-a35a-1815834d15da admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:26.833664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416231ee-ac8e-493b-a35a-1815834d15da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:33:26.833984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416231ee-ac8e-493b-a35a-1815834d15da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:26.835986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416231ee-ac8e-493b-a35a-1815834d15da admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:26.847430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1358/2700] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:33:26.851644 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a5638-3d4d-4b1c-833d-de7de2167808 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:26.860837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a5638-3d4d-4b1c-833d-de7de2167808 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:26.866026 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d18b93-f871-429f-8364-6f8b0701d43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:26.866619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d18b93-f871-429f-8364-6f8b0701d43d admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:26.866901 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d18b93-f871-429f-8364-6f8b0701d43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:26.867177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d18b93-f871-429f-8364-6f8b0701d43d admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:26.867806 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d18b93-f871-429f-8364-6f8b0701d43d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:26.869726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d18b93-f871-429f-8364-6f8b0701d43d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:26.899551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1343/2701] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:33:26 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:26.900808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416231ee-ac8e-493b-a35a-1815834d15da admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:26.910861 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416231ee-ac8e-493b-a35a-1815834d15da admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:26.965894 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d898a-3ec2-4cde-a789-8d57e3c8cc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:26.966487 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d898a-3ec2-4cde-a789-8d57e3c8cc7b admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:26.966762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d898a-3ec2-4cde-a789-8d57e3c8cc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:26.967031 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d898a-3ec2-4cde-a789-8d57e3c8cc7b admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:26.967889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d898a-3ec2-4cde-a789-8d57e3c8cc7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:26.969788 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d898a-3ec2-4cde-a789-8d57e3c8cc7b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:26.990898 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1359/2702] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:33:26 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:26.995388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d18b93-f871-429f-8364-6f8b0701d43d admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:27.006098 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d18b93-f871-429f-8364-6f8b0701d43d admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:27.010683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9220588-fe00-4c95-9aa0-51493df06c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:27.011276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9220588-fe00-4c95-9aa0-51493df06c0f admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:27.011559 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9220588-fe00-4c95-9aa0-51493df06c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:27.011847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9220588-fe00-4c95-9aa0-51493df06c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:27.012497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9220588-fe00-4c95-9aa0-51493df06c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:27.014112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9220588-fe00-4c95-9aa0-51493df06c0f admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:27.024730 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1344/2703] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:27.028598 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d898a-3ec2-4cde-a789-8d57e3c8cc7b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:27.038865 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d898a-3ec2-4cde-a789-8d57e3c8cc7b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:27.043385 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7d5cc-e6f6-4f33-91d5-789fe3a5432e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:27.043968 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7d5cc-e6f6-4f33-91d5-789fe3a5432e admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:27.044244 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7d5cc-e6f6-4f33-91d5-789fe3a5432e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:27.044515 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7d5cc-e6f6-4f33-91d5-789fe3a5432e admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:27.045194 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7d5cc-e6f6-4f33-91d5-789fe3a5432e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:27.046755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7d5cc-e6f6-4f33-91d5-789fe3a5432e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:27.057775 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1360/2704] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:27.061861 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9220588-fe00-4c95-9aa0-51493df06c0f admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:27.072526 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9220588-fe00-4c95-9aa0-51493df06c0f admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:27.077269 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea793da-8b42-4121-b871-3cd3007f2a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:27.077918 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea793da-8b42-4121-b871-3cd3007f2a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:27.078204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea793da-8b42-4121-b871-3cd3007f2a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:27.078481 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea793da-8b42-4121-b871-3cd3007f2a54 admin admin] PATH_INFO: `/v3/projects/d73c25391e8044c2ac3678317021732d/users/5b723fcdba1d4b26bd0cf604d59f8d8c/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:27.093355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea793da-8b42-4121-b871-3cd3007f2a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d73c25391e8044c2ac3678317021732d, user_id=5b723fcdba1d4b26bd0cf604d59f8d8c, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:27.095978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea793da-8b42-4121-b871-3cd3007f2a54 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:27.105828 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:27.111126 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1345/2705] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:27 2022] PUT /identity/v3/projects/d73c25391e8044c2ac3678317021732d/users/5b723fcdba1d4b26bd0cf604d59f8d8c/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:27.116778 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee686a31-8b39-423c-9f4f-63227e244c0c None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:27.117320 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee686a31-8b39-423c-9f4f-63227e244c0c None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:27.117420 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee686a31-8b39-423c-9f4f-63227e244c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:27.135680 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-ee686a31-8b39-423c-9f4f-63227e244c0c None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:27.136938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-ee686a31-8b39-423c-9f4f-63227e244c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:27.195649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1361/2706] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:27.205342 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea793da-8b42-4121-b871-3cd3007f2a54 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:27.207038 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7ea793da-8b42-4121-b871-3cd3007f2a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:27.264854 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea793da-8b42-4121-b871-3cd3007f2a54 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:27.268520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85ad96-9891-4c97-90e2-ab34e20f0815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:27.269146 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85ad96-9891-4c97-90e2-ab34e20f0815 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:27.269437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85ad96-9891-4c97-90e2-ab34e20f0815 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:27.269715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85ad96-9891-4c97-90e2-ab34e20f0815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:27.279059 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85ad96-9891-4c97-90e2-ab34e20f0815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:27.280403 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85ad96-9891-4c97-90e2-ab34e20f0815 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:27.290317 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1346/2707] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:31.308906 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88235157-5247-4ca7-83ce-f17fe726e7e6 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:31.309473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88235157-5247-4ca7-83ce-f17fe726e7e6 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:31.309473 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88235157-5247-4ca7-83ce-f17fe726e7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:31.337623 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-88235157-5247-4ca7-83ce-f17fe726e7e6 None None] MFA Rules not processed for user `5b723fcdba1d4b26bd0cf604d59f8d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:31.338824 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-88235157-5247-4ca7-83ce-f17fe726e7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:31.408474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:31.419332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1362/2708] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:31.423850 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85ad96-9891-4c97-90e2-ab34e20f0815 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:31.434828 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85ad96-9891-4c97-90e2-ab34e20f0815 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:31.440857 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e18a7-2b6c-4dbc-843b-ef9772b392e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:31.441520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e18a7-2b6c-4dbc-843b-ef9772b392e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:31.441786 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e18a7-2b6c-4dbc-843b-ef9772b392e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:31.442063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e18a7-2b6c-4dbc-843b-ef9772b392e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:31.443367 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e18a7-2b6c-4dbc-843b-ef9772b392e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:31.449698 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e18a7-2b6c-4dbc-843b-ef9772b392e5 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:31.494275 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1347/2709] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:33:31 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:31.498291 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88235157-5247-4ca7-83ce-f17fe726e7e6 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:31.511297 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88235157-5247-4ca7-83ce-f17fe726e7e6 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:31.589287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9d3d3-e508-47b6-9434-a5351eb07688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:31.589950 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9d3d3-e508-47b6-9434-a5351eb07688 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:31.590493 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9d3d3-e508-47b6-9434-a5351eb07688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:31.590833 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9d3d3-e508-47b6-9434-a5351eb07688 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:31.591567 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9d3d3-e508-47b6-9434-a5351eb07688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:31.593825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9d3d3-e508-47b6-9434-a5351eb07688 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:31.625741 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e18a7-2b6c-4dbc-843b-ef9772b392e5 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:31.629403 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1363/2710] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:33:31 2022] POST /identity/v3/users => generated 296 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:31.638015 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e18a7-2b6c-4dbc-843b-ef9772b392e5 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:31.642957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54203796-c622-4f68-8f06-b0bdf402d4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:31.643410 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54203796-c622-4f68-8f06-b0bdf402d4b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:31.643689 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54203796-c622-4f68-8f06-b0bdf402d4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:31.643960 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54203796-c622-4f68-8f06-b0bdf402d4b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:31.644622 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54203796-c622-4f68-8f06-b0bdf402d4b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:31.646432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54203796-c622-4f68-8f06-b0bdf402d4b1 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:31.658718 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1348/2711] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:31.663889 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9d3d3-e508-47b6-9434-a5351eb07688 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:31.674938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9d3d3-e508-47b6-9434-a5351eb07688 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:31.680752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0036d-dd41-449c-8282-2e0cd24d3480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:31.681456 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0036d-dd41-449c-8282-2e0cd24d3480 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:31.681733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0036d-dd41-449c-8282-2e0cd24d3480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:31.682008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0036d-dd41-449c-8282-2e0cd24d3480 admin admin] PATH_INFO: `/v3/projects/f621e8defb6c4a7796f19073bfcc35cb/users/e16f479e1cbf46068e05d1c60de13152/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:31.701737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0036d-dd41-449c-8282-2e0cd24d3480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f621e8defb6c4a7796f19073bfcc35cb, user_id=e16f479e1cbf46068e05d1c60de13152, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:31.706224 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0036d-dd41-449c-8282-2e0cd24d3480 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:31.724759 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1364/2712] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:31 2022] PUT /identity/v3/projects/f621e8defb6c4a7796f19073bfcc35cb/users/e16f479e1cbf46068e05d1c60de13152/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:31.728974 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54203796-c622-4f68-8f06-b0bdf402d4b1 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:31.740155 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54203796-c622-4f68-8f06-b0bdf402d4b1 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:31.806743 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b8fbb-0880-4c73-ba23-9d8b0fbdb89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:31.807399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b8fbb-0880-4c73-ba23-9d8b0fbdb89a admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:31.807764 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b8fbb-0880-4c73-ba23-9d8b0fbdb89a admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:31.808101 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b8fbb-0880-4c73-ba23-9d8b0fbdb89a admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:31.808807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b8fbb-0880-4c73-ba23-9d8b0fbdb89a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:31.810629 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b8fbb-0880-4c73-ba23-9d8b0fbdb89a admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:31.821920 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1349/2713] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:33:31.825935 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0036d-dd41-449c-8282-2e0cd24d3480 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:31.837543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0036d-dd41-449c-8282-2e0cd24d3480 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:31.843278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d85841-a9d5-4810-ae2b-47edf0043a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:31.843963 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d85841-a9d5-4810-ae2b-47edf0043a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:31.844240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d85841-a9d5-4810-ae2b-47edf0043a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:31.844512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d85841-a9d5-4810-ae2b-47edf0043a17 admin admin] PATH_INFO: `/v3/projects/f621e8defb6c4a7796f19073bfcc35cb/users/e16f479e1cbf46068e05d1c60de13152/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:31.847561 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d85841-a9d5-4810-ae2b-47edf0043a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f621e8defb6c4a7796f19073bfcc35cb, user_id=e16f479e1cbf46068e05d1c60de13152, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:31.850588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d85841-a9d5-4810-ae2b-47edf0043a17 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:31.865701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1365/2714] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:31 2022] PUT /identity/v3/projects/f621e8defb6c4a7796f19073bfcc35cb/users/e16f479e1cbf46068e05d1c60de13152/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:33:33.037751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b8fbb-0880-4c73-ba23-9d8b0fbdb89a admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:33.039624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5e3b8fbb-0880-4c73-ba23-9d8b0fbdb89a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:33.055710 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b8fbb-0880-4c73-ba23-9d8b0fbdb89a admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:33.147952 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b757d2-1dcc-444d-8355-551459bd66ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:33.149499 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b757d2-1dcc-444d-8355-551459bd66ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:33.149899 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b757d2-1dcc-444d-8355-551459bd66ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:33.150274 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b757d2-1dcc-444d-8355-551459bd66ff admin admin] PATH_INFO: `/v3/users/f6141a78337c4ab3b22e3e618449221c` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:33.151780 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b757d2-1dcc-444d-8355-551459bd66ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6141a78337c4ab3b22e3e618449221c)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:33.153883 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b757d2-1dcc-444d-8355-551459bd66ff admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:33.162242 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:33.230327 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f2b757d2-1dcc-444d-8355-551459bd66ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6141a78337c4ab3b22e3e618449221c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:33.231178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f2b757d2-1dcc-444d-8355-551459bd66ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6141a78337c4ab3b22e3e618449221c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:33.248508 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f2b757d2-1dcc-444d-8355-551459bd66ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6141a78337c4ab3b22e3e618449221c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:33.259553 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f2b757d2-1dcc-444d-8355-551459bd66ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6141a78337c4ab3b22e3e618449221c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:33.260446 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f2b757d2-1dcc-444d-8355-551459bd66ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6141a78337c4ab3b22e3e618449221c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:33.281215 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1350/2715] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:33:33 2022] DELETE /identity/v3/users/f6141a78337c4ab3b22e3e618449221c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:33.282912 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13070f-e1e2-4f00-a5f3-f52cac70870e None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:33.283349 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13070f-e1e2-4f00-a5f3-f52cac70870e None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:33.283992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13070f-e1e2-4f00-a5f3-f52cac70870e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:33.314113 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-2d13070f-e1e2-4f00-a5f3-f52cac70870e None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:33.315375 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2d13070f-e1e2-4f00-a5f3-f52cac70870e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:33.396306 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1366/2716] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:33.407753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b757d2-1dcc-444d-8355-551459bd66ff admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:33.409305 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f2b757d2-1dcc-444d-8355-551459bd66ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:33.484062 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b757d2-1dcc-444d-8355-551459bd66ff admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:33.488742 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb478f-b7c6-40f5-b45b-a0c58abceac0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:33.489690 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb478f-b7c6-40f5-b45b-a0c58abceac0 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:33.489991 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb478f-b7c6-40f5-b45b-a0c58abceac0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:33.490372 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb478f-b7c6-40f5-b45b-a0c58abceac0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:33.505362 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb478f-b7c6-40f5-b45b-a0c58abceac0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:33.508338 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb478f-b7c6-40f5-b45b-a0c58abceac0 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:33.523764 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1351/2717] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:33.885640 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13070f-e1e2-4f00-a5f3-f52cac70870e None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:33.887157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2d13070f-e1e2-4f00-a5f3-f52cac70870e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:33.922854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13070f-e1e2-4f00-a5f3-f52cac70870e None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:33.938307 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8466f42-22c7-47a7-adc1-8afd213cfa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:33.940616 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8466f42-22c7-47a7-adc1-8afd213cfa85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:33.941115 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8466f42-22c7-47a7-adc1-8afd213cfa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:33.943376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8466f42-22c7-47a7-adc1-8afd213cfa85 admin admin] PATH_INFO: `/v3/projects/40ef41cfe9d042f98b5da706c7743fb0` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:33.947049 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8466f42-22c7-47a7-adc1-8afd213cfa85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40ef41cfe9d042f98b5da706c7743fb0)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:33.964213 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8466f42-22c7-47a7-adc1-8afd213cfa85 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:34.045058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:34.059053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c8466f42-22c7-47a7-adc1-8afd213cfa85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40ef41cfe9d042f98b5da706c7743fb0', 'request_id': 'req-c8466f42-22c7-47a7-adc1-8afd213cfa85'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:34.072157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c8466f42-22c7-47a7-adc1-8afd213cfa85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40ef41cfe9d042f98b5da706c7743fb0', 'request_id': 'req-c8466f42-22c7-47a7-adc1-8afd213cfa85'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:34.102201 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c8466f42-22c7-47a7-adc1-8afd213cfa85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40ef41cfe9d042f98b5da706c7743fb0', 'request_id': 'req-c8466f42-22c7-47a7-adc1-8afd213cfa85'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:34.119937 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c8466f42-22c7-47a7-adc1-8afd213cfa85 admin admin] The token cache is being invalidate because project 40ef41cfe9d042f98b5da706c7743fb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:33:34.120240 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c8466f42-22c7-47a7-adc1-8afd213cfa85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:34.120887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-c8466f42-22c7-47a7-adc1-8afd213cfa85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:34.123939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1367/2718] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:33:33 2022] DELETE /identity/v3/projects/40ef41cfe9d042f98b5da706c7743fb0 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:34.126707 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb478f-b7c6-40f5-b45b-a0c58abceac0 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:34.138430 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb478f-b7c6-40f5-b45b-a0c58abceac0 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:34.262511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43887477-f59f-449f-9102-40daa4afe6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:34.263430 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43887477-f59f-449f-9102-40daa4afe6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:34.263895 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43887477-f59f-449f-9102-40daa4afe6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:34.264329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43887477-f59f-449f-9102-40daa4afe6de admin admin] PATH_INFO: `/v3/users/9af29509560347ce938e8a1a10714a49` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:34.267359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43887477-f59f-449f-9102-40daa4afe6de admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af29509560347ce938e8a1a10714a49)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:34.272708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43887477-f59f-449f-9102-40daa4afe6de admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:34.333607 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:34.357763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-43887477-f59f-449f-9102-40daa4afe6de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af29509560347ce938e8a1a10714a49'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:34.358506 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-43887477-f59f-449f-9102-40daa4afe6de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af29509560347ce938e8a1a10714a49'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:34.384729 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-43887477-f59f-449f-9102-40daa4afe6de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af29509560347ce938e8a1a10714a49'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:34.396163 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-43887477-f59f-449f-9102-40daa4afe6de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af29509560347ce938e8a1a10714a49'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:34.396837 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-43887477-f59f-449f-9102-40daa4afe6de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af29509560347ce938e8a1a10714a49'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:34.426557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1352/2719] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:33:34 2022] DELETE /identity/v3/users/9af29509560347ce938e8a1a10714a49 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:34.849555 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8466f42-22c7-47a7-adc1-8afd213cfa85 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:34.851735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-c8466f42-22c7-47a7-adc1-8afd213cfa85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:34.871199 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8466f42-22c7-47a7-adc1-8afd213cfa85 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:35.012824 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2bedb-ec29-4fb9-b6e0-824f6aaab0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:35.013511 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2bedb-ec29-4fb9-b6e0-824f6aaab0ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:35.013806 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2bedb-ec29-4fb9-b6e0-824f6aaab0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:35.014057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2bedb-ec29-4fb9-b6e0-824f6aaab0ba admin admin] PATH_INFO: `/v3/projects/b2fac19cacf944909ffeedc038b4a0ab` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:35.016479 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2bedb-ec29-4fb9-b6e0-824f6aaab0ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2fac19cacf944909ffeedc038b4a0ab)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:35.018574 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2bedb-ec29-4fb9-b6e0-824f6aaab0ba admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:35.123657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8dc2bedb-ec29-4fb9-b6e0-824f6aaab0ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2fac19cacf944909ffeedc038b4a0ab', 'request_id': 'req-8dc2bedb-ec29-4fb9-b6e0-824f6aaab0ba'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:35.138368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8dc2bedb-ec29-4fb9-b6e0-824f6aaab0ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2fac19cacf944909ffeedc038b4a0ab', 'request_id': 'req-8dc2bedb-ec29-4fb9-b6e0-824f6aaab0ba'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:35.158393 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8dc2bedb-ec29-4fb9-b6e0-824f6aaab0ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2fac19cacf944909ffeedc038b4a0ab', 'request_id': 'req-8dc2bedb-ec29-4fb9-b6e0-824f6aaab0ba'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:35.199334 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8dc2bedb-ec29-4fb9-b6e0-824f6aaab0ba admin admin] The token cache is being invalidate because project b2fac19cacf944909ffeedc038b4a0ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:33:35.199743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8dc2bedb-ec29-4fb9-b6e0-824f6aaab0ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:35.200369 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-8dc2bedb-ec29-4fb9-b6e0-824f6aaab0ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:35.202256 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1368/2720] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:33:34 2022] DELETE /identity/v3/projects/b2fac19cacf944909ffeedc038b4a0ab => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:35.206847 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239c089-947c-426e-98d7-8a6c9d3554aa None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:35.207861 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239c089-947c-426e-98d7-8a6c9d3554aa None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:35.208750 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239c089-947c-426e-98d7-8a6c9d3554aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:35.231579 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-3239c089-947c-426e-98d7-8a6c9d3554aa None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:35.233090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-3239c089-947c-426e-98d7-8a6c9d3554aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:35.324515 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab9574-aee9-416f-b58b-6b970ca7951f None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:35.324837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab9574-aee9-416f-b58b-6b970ca7951f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:35.325067 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab9574-aee9-416f-b58b-6b970ca7951f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:35.326426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1353/2721] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:35.356894 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-56ab9574-aee9-416f-b58b-6b970ca7951f None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:35.357894 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-56ab9574-aee9-416f-b58b-6b970ca7951f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:35.370472 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1369/2722] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:33:35.373078 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239c089-947c-426e-98d7-8a6c9d3554aa None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:35.387558 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239c089-947c-426e-98d7-8a6c9d3554aa None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:35.399882 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152773c9-a9ce-4910-81be-fb4f398fe624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:35.400852 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152773c9-a9ce-4910-81be-fb4f398fe624 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:35.401414 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152773c9-a9ce-4910-81be-fb4f398fe624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:35.401927 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152773c9-a9ce-4910-81be-fb4f398fe624 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:35.402831 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152773c9-a9ce-4910-81be-fb4f398fe624 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:33:35.403352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152773c9-a9ce-4910-81be-fb4f398fe624 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:35.405689 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152773c9-a9ce-4910-81be-fb4f398fe624 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:35.422245 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1354/2723] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:33:35.426247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab9574-aee9-416f-b58b-6b970ca7951f None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:35.437031 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab9574-aee9-416f-b58b-6b970ca7951f None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:35.443273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69efb54-0d11-443e-8586-409b220684a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:35.443949 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69efb54-0d11-443e-8586-409b220684a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:35.444223 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69efb54-0d11-443e-8586-409b220684a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:35.444545 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69efb54-0d11-443e-8586-409b220684a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:35.445924 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69efb54-0d11-443e-8586-409b220684a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:35.447225 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69efb54-0d11-443e-8586-409b220684a9 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:35.487543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1370/2724] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:33:35 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:33:35.490266 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152773c9-a9ce-4910-81be-fb4f398fe624 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:35.502307 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152773c9-a9ce-4910-81be-fb4f398fe624 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:35.580394 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a4556-8b04-47b8-9953-be01803bb13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:35.581418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a4556-8b04-47b8-9953-be01803bb13b admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:35.581781 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a4556-8b04-47b8-9953-be01803bb13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:35.582308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a4556-8b04-47b8-9953-be01803bb13b admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:35.583653 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a4556-8b04-47b8-9953-be01803bb13b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:35.585970 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a4556-8b04-47b8-9953-be01803bb13b admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:35.621974 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1355/2725] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:33:35 2022] POST /identity/v3/users => generated 291 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:35.625806 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69efb54-0d11-443e-8586-409b220684a9 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:35.640084 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69efb54-0d11-443e-8586-409b220684a9 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:35.647759 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e234192-da3c-467d-8496-e3c21e111f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:35.648376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e234192-da3c-467d-8496-e3c21e111f2f admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:35.648661 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e234192-da3c-467d-8496-e3c21e111f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:35.648931 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e234192-da3c-467d-8496-e3c21e111f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:35.649679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e234192-da3c-467d-8496-e3c21e111f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:35.651270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e234192-da3c-467d-8496-e3c21e111f2f admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:35.666994 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1371/2726] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:33:35.668245 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a4556-8b04-47b8-9953-be01803bb13b admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:35.684418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a4556-8b04-47b8-9953-be01803bb13b admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:35.689259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7eaaa-1978-429e-aa18-3491e6b33f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:35.689935 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7eaaa-1978-429e-aa18-3491e6b33f42 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:35.690219 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7eaaa-1978-429e-aa18-3491e6b33f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:35.690489 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7eaaa-1978-429e-aa18-3491e6b33f42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:35.691149 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7eaaa-1978-429e-aa18-3491e6b33f42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:35.693328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7eaaa-1978-429e-aa18-3491e6b33f42 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:35.709946 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e234192-da3c-467d-8496-e3c21e111f2f admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:35.717298 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1356/2727] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:35.722395 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e234192-da3c-467d-8496-e3c21e111f2f admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:35.729826 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71eed84-df93-45c9-aa68-97127d9fbf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:35.731222 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71eed84-df93-45c9-aa68-97127d9fbf33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:35.731559 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71eed84-df93-45c9-aa68-97127d9fbf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:35.731854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71eed84-df93-45c9-aa68-97127d9fbf33 admin admin] PATH_INFO: `/v3/projects/ddab892310c647548a06cc404095fede/users/abc3d84041144c7797dab71de05f4b1b/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:35.752689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71eed84-df93-45c9-aa68-97127d9fbf33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddab892310c647548a06cc404095fede, user_id=abc3d84041144c7797dab71de05f4b1b, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:35.756694 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71eed84-df93-45c9-aa68-97127d9fbf33 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:35.761495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:35.780805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1372/2728] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:35 2022] PUT /identity/v3/projects/ddab892310c647548a06cc404095fede/users/abc3d84041144c7797dab71de05f4b1b/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:35.791512 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77e9c8-7f4a-4899-96ec-913a3e31172f None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:35.792405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77e9c8-7f4a-4899-96ec-913a3e31172f None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:35.792694 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77e9c8-7f4a-4899-96ec-913a3e31172f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:35.830569 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:35.837484 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-6d77e9c8-7f4a-4899-96ec-913a3e31172f None None] MFA Rules not processed for user `abc3d84041144c7797dab71de05f4b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:35.841631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-6d77e9c8-7f4a-4899-96ec-913a3e31172f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:35.977968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1357/2729] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:35.983556 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71eed84-df93-45c9-aa68-97127d9fbf33 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:36.104025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71eed84-df93-45c9-aa68-97127d9fbf33 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:36.109615 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e563b4-2f2b-43ee-8022-cd442efddee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:36.111123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e563b4-2f2b-43ee-8022-cd442efddee6 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:36.111123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e563b4-2f2b-43ee-8022-cd442efddee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:36.111123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e563b4-2f2b-43ee-8022-cd442efddee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:36.113023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e563b4-2f2b-43ee-8022-cd442efddee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:36.115054 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e563b4-2f2b-43ee-8022-cd442efddee6 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:36.156820 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77e9c8-7f4a-4899-96ec-913a3e31172f None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:36.162194 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1373/2730] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:33:35 2022] POST /identity/v3/projects => generated 381 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:36.170417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77e9c8-7f4a-4899-96ec-913a3e31172f None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:36.305475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba94884-6e36-4925-8be9-8765d54f48f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:36.306178 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba94884-6e36-4925-8be9-8765d54f48f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:36.307113 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba94884-6e36-4925-8be9-8765d54f48f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:36.307407 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba94884-6e36-4925-8be9-8765d54f48f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:36.308214 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba94884-6e36-4925-8be9-8765d54f48f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:36.311063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba94884-6e36-4925-8be9-8765d54f48f5 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:36.360583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1358/2731] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:33:36 2022] POST /identity/v3/users => generated 297 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:36.364604 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e563b4-2f2b-43ee-8022-cd442efddee6 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:36.377594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e563b4-2f2b-43ee-8022-cd442efddee6 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:36.390624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb4844-f4e6-452b-9e95-544c699ccf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:36.391481 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb4844-f4e6-452b-9e95-544c699ccf0d admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:36.392118 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb4844-f4e6-452b-9e95-544c699ccf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:36.392532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb4844-f4e6-452b-9e95-544c699ccf0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:36.393391 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb4844-f4e6-452b-9e95-544c699ccf0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:36.395168 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb4844-f4e6-452b-9e95-544c699ccf0d admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:36.415732 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1374/2732] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:36.419643 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba94884-6e36-4925-8be9-8765d54f48f5 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:36.440376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba94884-6e36-4925-8be9-8765d54f48f5 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:36.447110 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9318eb-0e34-4e18-9088-822dc1ea02b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:36.447794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9318eb-0e34-4e18-9088-822dc1ea02b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:36.448078 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9318eb-0e34-4e18-9088-822dc1ea02b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:36.448353 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9318eb-0e34-4e18-9088-822dc1ea02b1 admin admin] PATH_INFO: `/v3/projects/2decc4e11eba45e99d98e5f2a113f682/users/4da5a69574e94225b7eb0cf35323a10e/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:36.474903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9318eb-0e34-4e18-9088-822dc1ea02b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2decc4e11eba45e99d98e5f2a113f682, user_id=4da5a69574e94225b7eb0cf35323a10e, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:36.477900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9318eb-0e34-4e18-9088-822dc1ea02b1 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:36.504308 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1359/2733] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:36 2022] PUT /identity/v3/projects/2decc4e11eba45e99d98e5f2a113f682/users/4da5a69574e94225b7eb0cf35323a10e/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:36.506347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb4844-f4e6-452b-9e95-544c699ccf0d admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:36.520007 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb4844-f4e6-452b-9e95-544c699ccf0d admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:36.615710 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7081b46-336c-4f88-b90c-395de17f55d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:36.618028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7081b46-336c-4f88-b90c-395de17f55d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:36.618532 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7081b46-336c-4f88-b90c-395de17f55d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:36.620023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7081b46-336c-4f88-b90c-395de17f55d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:36.620786 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7081b46-336c-4f88-b90c-395de17f55d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:36.623332 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7081b46-336c-4f88-b90c-395de17f55d6 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:36.640213 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1375/2734] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:36.641587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9318eb-0e34-4e18-9088-822dc1ea02b1 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:36.655100 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9318eb-0e34-4e18-9088-822dc1ea02b1 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:36.671214 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc7a2c9-246b-470c-8984-c1399d09f266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:36.672038 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7a2c9-246b-470c-8984-c1399d09f266 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:36.672442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7a2c9-246b-470c-8984-c1399d09f266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:36.673535 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7a2c9-246b-470c-8984-c1399d09f266 admin admin] PATH_INFO: `/v3/projects/2decc4e11eba45e99d98e5f2a113f682/users/4da5a69574e94225b7eb0cf35323a10e/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:36.700265 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc7a2c9-246b-470c-8984-c1399d09f266 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2decc4e11eba45e99d98e5f2a113f682, user_id=4da5a69574e94225b7eb0cf35323a10e, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:36.702999 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc7a2c9-246b-470c-8984-c1399d09f266 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:36.729424 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1360/2735] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:36 2022] PUT /identity/v3/projects/2decc4e11eba45e99d98e5f2a113f682/users/4da5a69574e94225b7eb0cf35323a10e/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:36.741320 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f694f1-21e4-4e1d-b685-e1f9ecbee7cf None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:36.741612 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f694f1-21e4-4e1d-b685-e1f9ecbee7cf None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:36.742200 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f694f1-21e4-4e1d-b685-e1f9ecbee7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:36.773869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-b4f694f1-21e4-4e1d-b685-e1f9ecbee7cf None None] MFA Rules not processed for user `4da5a69574e94225b7eb0cf35323a10e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:36.775206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b4f694f1-21e4-4e1d-b685-e1f9ecbee7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:36.831304 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:36.860855 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1376/2736] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:36.869626 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc7a2c9-246b-470c-8984-c1399d09f266 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:36.871460 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1cc7a2c9-246b-470c-8984-c1399d09f266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:36.938094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc7a2c9-246b-470c-8984-c1399d09f266 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:36.942043 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910de260-cd8f-45f0-8d7c-e32817e927f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:36.942797 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910de260-cd8f-45f0-8d7c-e32817e927f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:36.943150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910de260-cd8f-45f0-8d7c-e32817e927f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:36.943430 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910de260-cd8f-45f0-8d7c-e32817e927f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:36.953916 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910de260-cd8f-45f0-8d7c-e32817e927f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:36.955690 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910de260-cd8f-45f0-8d7c-e32817e927f7 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:36.962008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:36.971474 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1361/2737] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:36 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:37.305267 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02c614-a510-40a8-9841-88cabb793e9e None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:37.305804 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02c614-a510-40a8-9841-88cabb793e9e None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:37.306002 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02c614-a510-40a8-9841-88cabb793e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:37.337354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-9d02c614-a510-40a8-9841-88cabb793e9e None None] MFA Rules not processed for user `e16f479e1cbf46068e05d1c60de13152`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:37.338666 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9d02c614-a510-40a8-9841-88cabb793e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:37.414286 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910de260-cd8f-45f0-8d7c-e32817e927f7 service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:37.424283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910de260-cd8f-45f0-8d7c-e32817e927f7 service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:37.431823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1377/2738] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:33:37.432226 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c6775-97e5-406f-93d2-99c861a40a72 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da5a69574e94225b7eb0cf35323a10e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2decc4e11eba45e99d98e5f2a113f682', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:37.432849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c6775-97e5-406f-93d2-99c861a40a72 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:37.433152 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c6775-97e5-406f-93d2-99c861a40a72 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:37.433432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c6775-97e5-406f-93d2-99c861a40a72 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:37.434078 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c6775-97e5-406f-93d2-99c861a40a72 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:33:37.434529 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c6775-97e5-406f-93d2-99c861a40a72 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:37.436482 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c6775-97e5-406f-93d2-99c861a40a72 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:37.446915 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02c614-a510-40a8-9841-88cabb793e9e None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:37.448309 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9d02c614-a510-40a8-9841-88cabb793e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:37.451019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1362/2739] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:33:37.456298 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c6775-97e5-406f-93d2-99c861a40a72 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:37.467554 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c6775-97e5-406f-93d2-99c861a40a72 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:37.474127 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e5720-bc49-4b5e-b45a-1c52652f9395 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da5a69574e94225b7eb0cf35323a10e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2decc4e11eba45e99d98e5f2a113f682', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:37.474817 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e5720-bc49-4b5e-b45a-1c52652f9395 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:37.475102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e5720-bc49-4b5e-b45a-1c52652f9395 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:37.475373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e5720-bc49-4b5e-b45a-1c52652f9395 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:37.475994 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e5720-bc49-4b5e-b45a-1c52652f9395 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:37.477908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e5720-bc49-4b5e-b45a-1c52652f9395 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:37.517219 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1363/2740] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:33:37 2022] POST /identity/v3/projects => generated 352 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:37.521517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e5720-bc49-4b5e-b45a-1c52652f9395 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:37.541496 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02c614-a510-40a8-9841-88cabb793e9e None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:37.599475 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e5720-bc49-4b5e-b45a-1c52652f9395 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:37.604426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02d37a-1919-434f-abf9-2b735f949ffa tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da5a69574e94225b7eb0cf35323a10e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2decc4e11eba45e99d98e5f2a113f682', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:37.605280 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d37a-1919-434f-abf9-2b735f949ffa tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:37.605636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d37a-1919-434f-abf9-2b735f949ffa tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:37.605918 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d37a-1919-434f-abf9-2b735f949ffa tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:37.606566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02d37a-1919-434f-abf9-2b735f949ffa tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:33:37.606871 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02d37a-1919-434f-abf9-2b735f949ffa tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:37.608473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02d37a-1919-434f-abf9-2b735f949ffa tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:37.613042 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7a4e7-e181-4f8d-9124-42d8f5e915fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:37.613683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7a4e7-e181-4f8d-9124-42d8f5e915fd service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:37.613961 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7a4e7-e181-4f8d-9124-42d8f5e915fd service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:37.615144 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7a4e7-e181-4f8d-9124-42d8f5e915fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:37.617076 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-f6f7a4e7-e181-4f8d-9124-42d8f5e915fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:37.624774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1364/2741] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:33:37.628527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=100 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:37.660668 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02d37a-1919-434f-abf9-2b735f949ffa tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:37.671515 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02d37a-1919-434f-abf9-2b735f949ffa tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:37.676735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5cba7-2414-4de1-b61b-f1b5ca2df201 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da5a69574e94225b7eb0cf35323a10e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2decc4e11eba45e99d98e5f2a113f682', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:37.677426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5cba7-2414-4de1-b61b-f1b5ca2df201 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:37.678109 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5cba7-2414-4de1-b61b-f1b5ca2df201 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:37.678388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5cba7-2414-4de1-b61b-f1b5ca2df201 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] PATH_INFO: `/v3/projects/807c201a500346849057e61565772d66` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:37.679883 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf5cba7-2414-4de1-b61b-f1b5ca2df201 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] RBAC: Authorizing `identity:delete_project(project_id=807c201a500346849057e61565772d66)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:37.681863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf5cba7-2414-4de1-b61b-f1b5ca2df201 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:37.701641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7a4e7-e181-4f8d-9124-42d8f5e915fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:37.705243 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7a4e7-e181-4f8d-9124-42d8f5e915fd service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:37.716182 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1378/2742] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:37.761884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-4cf5cba7-2414-4de1-b61b-f1b5ca2df201 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '807c201a500346849057e61565772d66', 'request_id': 'req-4cf5cba7-2414-4de1-b61b-f1b5ca2df201'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:37.777477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-4cf5cba7-2414-4de1-b61b-f1b5ca2df201 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '807c201a500346849057e61565772d66', 'request_id': 'req-4cf5cba7-2414-4de1-b61b-f1b5ca2df201'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:37.802231 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-4cf5cba7-2414-4de1-b61b-f1b5ca2df201 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '807c201a500346849057e61565772d66', 'request_id': 'req-4cf5cba7-2414-4de1-b61b-f1b5ca2df201'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:37.818373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-4cf5cba7-2414-4de1-b61b-f1b5ca2df201 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] The token cache is being invalidate because project 807c201a500346849057e61565772d66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:33:37.818373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-4cf5cba7-2414-4de1-b61b-f1b5ca2df201 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:37.819246 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-4cf5cba7-2414-4de1-b61b-f1b5ca2df201 tempest-VolumeQuotaClassesTest-2044816230 tempest-VolumeQuotaClassesTest-2044816230-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:37.825647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1365/2743] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:33:37 2022] DELETE /identity/v3/projects/807c201a500346849057e61565772d66 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:33:38.074390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7a4e7-e181-4f8d-9124-42d8f5e915fd service nova] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:38.163188 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7a4e7-e181-4f8d-9124-42d8f5e915fd service nova] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:38.256565 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d64df-2f05-4a22-892d-33466055e3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:38.257281 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d64df-2f05-4a22-892d-33466055e3ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:38.257584 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d64df-2f05-4a22-892d-33466055e3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:38.257845 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d64df-2f05-4a22-892d-33466055e3ed admin admin] PATH_INFO: `/v3/users/abc3d84041144c7797dab71de05f4b1b` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:38.259082 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d64df-2f05-4a22-892d-33466055e3ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=abc3d84041144c7797dab71de05f4b1b)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:38.261010 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d64df-2f05-4a22-892d-33466055e3ed admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:38.330534 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-781d64df-2f05-4a22-892d-33466055e3ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abc3d84041144c7797dab71de05f4b1b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:38.331281 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-781d64df-2f05-4a22-892d-33466055e3ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abc3d84041144c7797dab71de05f4b1b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:38.340564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-781d64df-2f05-4a22-892d-33466055e3ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abc3d84041144c7797dab71de05f4b1b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:38.341217 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-781d64df-2f05-4a22-892d-33466055e3ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abc3d84041144c7797dab71de05f4b1b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:38.347717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:38.350847 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-781d64df-2f05-4a22-892d-33466055e3ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abc3d84041144c7797dab71de05f4b1b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:38.375838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1379/2744] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:33:38 2022] DELETE /identity/v3/users/abc3d84041144c7797dab71de05f4b1b => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:38.380339 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c1639-b1ad-4f6b-bb50-e94f1a6f236e None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:38.380715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c1639-b1ad-4f6b-bb50-e94f1a6f236e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:38.381060 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c1639-b1ad-4f6b-bb50-e94f1a6f236e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:38.404849 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e86c1639-b1ad-4f6b-bb50-e94f1a6f236e None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:38.406232 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e86c1639-b1ad-4f6b-bb50-e94f1a6f236e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:38.480569 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:38.536696 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1366/2745] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:33:38.537778 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d64df-2f05-4a22-892d-33466055e3ed admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:38.540433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-781d64df-2f05-4a22-892d-33466055e3ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:38.633345 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d64df-2f05-4a22-892d-33466055e3ed admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:38.640422 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54ba88-a825-43f7-b0ee-6ae3964700f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:38.641205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54ba88-a825-43f7-b0ee-6ae3964700f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:38.641559 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54ba88-a825-43f7-b0ee-6ae3964700f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:38.641961 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54ba88-a825-43f7-b0ee-6ae3964700f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:38.665222 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54ba88-a825-43f7-b0ee-6ae3964700f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:38.666848 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54ba88-a825-43f7-b0ee-6ae3964700f2 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:38.680510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1380/2746] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:39.143949 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c1639-b1ad-4f6b-bb50-e94f1a6f236e None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:39.145774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e86c1639-b1ad-4f6b-bb50-e94f1a6f236e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:39.164264 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c1639-b1ad-4f6b-bb50-e94f1a6f236e None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:39.174536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:39.180566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:39.180998 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:39.181202 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56 admin admin] PATH_INFO: `/v3/projects/ddab892310c647548a06cc404095fede` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:39.182570 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddab892310c647548a06cc404095fede)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:39.184567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:39.253015 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddab892310c647548a06cc404095fede', 'request_id': 'req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:39.276376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddab892310c647548a06cc404095fede', 'request_id': 'req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:39.297966 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddab892310c647548a06cc404095fede', 'request_id': 'req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:39.311962 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56 admin admin] The token cache is being invalidate because project ddab892310c647548a06cc404095fede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:33:39.311962 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:39.312772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:39.314708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1367/2747] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:33:39 2022] DELETE /identity/v3/projects/ddab892310c647548a06cc404095fede => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:39.319250 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54ba88-a825-43f7-b0ee-6ae3964700f2 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:39.330972 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54ba88-a825-43f7-b0ee-6ae3964700f2 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:39.397925 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6820c-c15e-41ad-8258-64af932cc153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:39.398653 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6820c-c15e-41ad-8258-64af932cc153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:39.398935 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6820c-c15e-41ad-8258-64af932cc153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:39.399204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6820c-c15e-41ad-8258-64af932cc153 admin admin] PATH_INFO: `/v3/users/4da5a69574e94225b7eb0cf35323a10e` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:39.400315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6820c-c15e-41ad-8258-64af932cc153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4da5a69574e94225b7eb0cf35323a10e)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:39.402241 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6820c-c15e-41ad-8258-64af932cc153 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:39.450368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-66e6820c-c15e-41ad-8258-64af932cc153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4da5a69574e94225b7eb0cf35323a10e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:39.451373 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-66e6820c-c15e-41ad-8258-64af932cc153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4da5a69574e94225b7eb0cf35323a10e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:39.459113 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-66e6820c-c15e-41ad-8258-64af932cc153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4da5a69574e94225b7eb0cf35323a10e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:39.459918 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-66e6820c-c15e-41ad-8258-64af932cc153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4da5a69574e94225b7eb0cf35323a10e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:39.468041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-66e6820c-c15e-41ad-8258-64af932cc153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4da5a69574e94225b7eb0cf35323a10e'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:39.482756 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:39.487902 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1381/2748] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:33:39 2022] DELETE /identity/v3/users/4da5a69574e94225b7eb0cf35323a10e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:39.805937 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:39.807459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:39.820626 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2ccfb-f5cd-4f79-bc9f-5c2304958c56 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:39.889798 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0079bca-c57d-46ea-88f7-35cb0e429db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:39.890436 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0079bca-c57d-46ea-88f7-35cb0e429db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:39.890718 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0079bca-c57d-46ea-88f7-35cb0e429db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:39.890991 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0079bca-c57d-46ea-88f7-35cb0e429db0 admin admin] PATH_INFO: `/v3/projects/2decc4e11eba45e99d98e5f2a113f682` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:39.892315 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0079bca-c57d-46ea-88f7-35cb0e429db0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2decc4e11eba45e99d98e5f2a113f682)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:39.894409 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0079bca-c57d-46ea-88f7-35cb0e429db0 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:39.950647 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-c0079bca-c57d-46ea-88f7-35cb0e429db0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2decc4e11eba45e99d98e5f2a113f682', 'request_id': 'req-c0079bca-c57d-46ea-88f7-35cb0e429db0'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:39.960939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-c0079bca-c57d-46ea-88f7-35cb0e429db0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2decc4e11eba45e99d98e5f2a113f682', 'request_id': 'req-c0079bca-c57d-46ea-88f7-35cb0e429db0'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:39.970148 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-c0079bca-c57d-46ea-88f7-35cb0e429db0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2decc4e11eba45e99d98e5f2a113f682', 'request_id': 'req-c0079bca-c57d-46ea-88f7-35cb0e429db0'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:39.975792 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:39.992871 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-c0079bca-c57d-46ea-88f7-35cb0e429db0 admin admin] The token cache is being invalidate because project 2decc4e11eba45e99d98e5f2a113f682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:33:39.993191 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-c0079bca-c57d-46ea-88f7-35cb0e429db0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:39.996629 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-c0079bca-c57d-46ea-88f7-35cb0e429db0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:40.002046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1368/2749] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:33:39 2022] DELETE /identity/v3/projects/2decc4e11eba45e99d98e5f2a113f682 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:40.003756 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a8a38-5235-471c-91aa-b32f503b8920 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:40.004043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a8a38-5235-471c-91aa-b32f503b8920 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:40.004317 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a8a38-5235-471c-91aa-b32f503b8920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:40.052465 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-203a8a38-5235-471c-91aa-b32f503b8920 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:40.053824 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-203a8a38-5235-471c-91aa-b32f503b8920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:40.127298 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1382/2750] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:33:40.134681 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d2ce5-a01c-4f7a-b9a9-543df7b82a6c None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:40.135321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d2ce5-a01c-4f7a-b9a9-543df7b82a6c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:40.135640 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d2ce5-a01c-4f7a-b9a9-543df7b82a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:40.161479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-888d2ce5-a01c-4f7a-b9a9-543df7b82a6c None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:40.162873 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-888d2ce5-a01c-4f7a-b9a9-543df7b82a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:40.177020 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1369/2751] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:40.181380 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a8a38-5235-471c-91aa-b32f503b8920 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:40.191560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a8a38-5235-471c-91aa-b32f503b8920 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:40.197466 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de299ff5-37de-4a68-9fcb-92c742b138dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:40.198162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de299ff5-37de-4a68-9fcb-92c742b138dd admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:40.198439 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de299ff5-37de-4a68-9fcb-92c742b138dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:40.198708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de299ff5-37de-4a68-9fcb-92c742b138dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:40.199383 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de299ff5-37de-4a68-9fcb-92c742b138dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:33:40.199739 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de299ff5-37de-4a68-9fcb-92c742b138dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:40.201348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de299ff5-37de-4a68-9fcb-92c742b138dd admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:40.217174 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1383/2752] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:33:40.219526 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d2ce5-a01c-4f7a-b9a9-543df7b82a6c None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:40.230233 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d2ce5-a01c-4f7a-b9a9-543df7b82a6c None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:40.235273 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3b506-7f87-4861-b7ed-89ecebc6e2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:40.235850 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3b506-7f87-4861-b7ed-89ecebc6e2f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:40.236172 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3b506-7f87-4861-b7ed-89ecebc6e2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:40.236401 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3b506-7f87-4861-b7ed-89ecebc6e2f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:40.237114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc3b506-7f87-4861-b7ed-89ecebc6e2f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:40.239341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc3b506-7f87-4861-b7ed-89ecebc6e2f0 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:40.276386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1370/2753] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:33:40 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:40.277620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de299ff5-37de-4a68-9fcb-92c742b138dd admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:40.292038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de299ff5-37de-4a68-9fcb-92c742b138dd admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:40.353728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c89b3-c898-4f96-8434-ea777b8ecc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:40.354444 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c89b3-c898-4f96-8434-ea777b8ecc4b admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:40.354615 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c89b3-c898-4f96-8434-ea777b8ecc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:40.354863 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c89b3-c898-4f96-8434-ea777b8ecc4b admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:40.355478 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c89b3-c898-4f96-8434-ea777b8ecc4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:40.357377 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c89b3-c898-4f96-8434-ea777b8ecc4b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:40.380456 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1384/2754] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:33:40 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:40.384480 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3b506-7f87-4861-b7ed-89ecebc6e2f0 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:40.396990 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3b506-7f87-4861-b7ed-89ecebc6e2f0 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:40.403481 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0d491-2404-4a45-af39-f4ab6c51a921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:40.403978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0d491-2404-4a45-af39-f4ab6c51a921 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:40.404288 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0d491-2404-4a45-af39-f4ab6c51a921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:40.404520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0d491-2404-4a45-af39-f4ab6c51a921 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:40.405182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0d491-2404-4a45-af39-f4ab6c51a921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:40.406785 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0d491-2404-4a45-af39-f4ab6c51a921 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:40.422232 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1371/2755] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:33:40.423462 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c89b3-c898-4f96-8434-ea777b8ecc4b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:40.435475 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c89b3-c898-4f96-8434-ea777b8ecc4b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:40.442086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90699a37-aae1-4271-929b-ec4bc0618548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:40.443084 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90699a37-aae1-4271-929b-ec4bc0618548 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:40.443447 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90699a37-aae1-4271-929b-ec4bc0618548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:40.443912 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90699a37-aae1-4271-929b-ec4bc0618548 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:40.444564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90699a37-aae1-4271-929b-ec4bc0618548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:40.446239 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90699a37-aae1-4271-929b-ec4bc0618548 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:40.458247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1385/2756] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:40.462631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0d491-2404-4a45-af39-f4ab6c51a921 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:40.474434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0d491-2404-4a45-af39-f4ab6c51a921 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:40.480979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc595d-f698-449a-bdb6-c12c37c236b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:40.481696 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc595d-f698-449a-bdb6-c12c37c236b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:40.481981 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc595d-f698-449a-bdb6-c12c37c236b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:40.482255 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc595d-f698-449a-bdb6-c12c37c236b9 admin admin] PATH_INFO: `/v3/projects/4b6771e4c5eb46b6a0cd0ccc065e9b88/users/508c3d45a6034ce088b77a622d4e4ef6/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:40.500571 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc595d-f698-449a-bdb6-c12c37c236b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b6771e4c5eb46b6a0cd0ccc065e9b88, user_id=508c3d45a6034ce088b77a622d4e4ef6, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:40.503303 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc595d-f698-449a-bdb6-c12c37c236b9 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:40.521001 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1372/2757] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:40 2022] PUT /identity/v3/projects/4b6771e4c5eb46b6a0cd0ccc065e9b88/users/508c3d45a6034ce088b77a622d4e4ef6/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:33:40.526011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b855eb7-d6fa-4c2c-8622-8766ba498a4a None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:40.526590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b855eb7-d6fa-4c2c-8622-8766ba498a4a None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:40.527238 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b855eb7-d6fa-4c2c-8622-8766ba498a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:40.552523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-9b855eb7-d6fa-4c2c-8622-8766ba498a4a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:40.554587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9b855eb7-d6fa-4c2c-8622-8766ba498a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:40.634778 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1386/2758] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:40.644323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc595d-f698-449a-bdb6-c12c37c236b9 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:40.647123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-dfbc595d-f698-449a-bdb6-c12c37c236b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:40.735276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc595d-f698-449a-bdb6-c12c37c236b9 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:40.742725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ae117-6a24-48c0-8e9e-91eca2439e5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:40.743528 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ae117-6a24-48c0-8e9e-91eca2439e5e service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:40.743811 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ae117-6a24-48c0-8e9e-91eca2439e5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:40.744085 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ae117-6a24-48c0-8e9e-91eca2439e5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:40.755447 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ae117-6a24-48c0-8e9e-91eca2439e5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:40.757110 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ae117-6a24-48c0-8e9e-91eca2439e5e service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:40.768589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1373/2759] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:41.548819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f86fd-1f0a-42c2-94de-f5ae9afbfca0 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:41.548819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f86fd-1f0a-42c2-94de-f5ae9afbfca0 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:41.548819 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f86fd-1f0a-42c2-94de-f5ae9afbfca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:41.569778 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:41.593851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-484f86fd-1f0a-42c2-94de-f5ae9afbfca0 None None] MFA Rules not processed for user `508c3d45a6034ce088b77a622d4e4ef6`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:41.595090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-484f86fd-1f0a-42c2-94de-f5ae9afbfca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:41.692079 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1387/2760] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:41.693600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ae117-6a24-48c0-8e9e-91eca2439e5e service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:41.704587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ae117-6a24-48c0-8e9e-91eca2439e5e service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:41.709699 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b26e9a-6cba-4c0e-9cdb-b75539ef9c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:41.710470 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b26e9a-6cba-4c0e-9cdb-b75539ef9c7e admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:41.710828 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b26e9a-6cba-4c0e-9cdb-b75539ef9c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:41.711177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b26e9a-6cba-4c0e-9cdb-b75539ef9c7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:41.711903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b26e9a-6cba-4c0e-9cdb-b75539ef9c7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:41.715707 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b26e9a-6cba-4c0e-9cdb-b75539ef9c7e admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:41.753184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1374/2761] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:33:41 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:41.758936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f86fd-1f0a-42c2-94de-f5ae9afbfca0 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:41.774560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f86fd-1f0a-42c2-94de-f5ae9afbfca0 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:41.855907 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646254eb-a783-4498-a838-85fc88e38f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:41.856538 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646254eb-a783-4498-a838-85fc88e38f24 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:41.856980 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646254eb-a783-4498-a838-85fc88e38f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:41.857522 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646254eb-a783-4498-a838-85fc88e38f24 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:41.858570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646254eb-a783-4498-a838-85fc88e38f24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:41.861060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646254eb-a783-4498-a838-85fc88e38f24 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:41.892187 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1388/2762] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:33:41 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:33:41.897037 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b26e9a-6cba-4c0e-9cdb-b75539ef9c7e admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:41.908546 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b26e9a-6cba-4c0e-9cdb-b75539ef9c7e admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:41.914545 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b08fd4-879e-46a9-b323-ab5490b7ed96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:41.915327 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b08fd4-879e-46a9-b323-ab5490b7ed96 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:41.915716 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b08fd4-879e-46a9-b323-ab5490b7ed96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:41.916072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b08fd4-879e-46a9-b323-ab5490b7ed96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:41.916747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b08fd4-879e-46a9-b323-ab5490b7ed96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:41.918506 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b08fd4-879e-46a9-b323-ab5490b7ed96 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:41.930770 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1375/2763] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:33:41.934936 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646254eb-a783-4498-a838-85fc88e38f24 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:41.945859 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646254eb-a783-4498-a838-85fc88e38f24 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:41.952126 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178aa07e-6044-4678-8289-d780f0213665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:41.952843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178aa07e-6044-4678-8289-d780f0213665 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:41.953182 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178aa07e-6044-4678-8289-d780f0213665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:41.953478 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178aa07e-6044-4678-8289-d780f0213665 admin admin] PATH_INFO: `/v3/projects/a98bccf3b93245dd950f6e1f1c924676/users/c4f4fd62f9b2498c9b5b5fc9de15f170/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:41.977471 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178aa07e-6044-4678-8289-d780f0213665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a98bccf3b93245dd950f6e1f1c924676, user_id=c4f4fd62f9b2498c9b5b5fc9de15f170, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:41.980297 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178aa07e-6044-4678-8289-d780f0213665 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:41.994368 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1389/2764] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:41 2022] PUT /identity/v3/projects/a98bccf3b93245dd950f6e1f1c924676/users/c4f4fd62f9b2498c9b5b5fc9de15f170/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:41.998461 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b08fd4-879e-46a9-b323-ab5490b7ed96 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:42.009232 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b08fd4-879e-46a9-b323-ab5490b7ed96 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:42.072659 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3597fbd-5403-4219-875e-59ac23c3d4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:42.073302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3597fbd-5403-4219-875e-59ac23c3d4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:42.073584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3597fbd-5403-4219-875e-59ac23c3d4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:42.073859 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3597fbd-5403-4219-875e-59ac23c3d4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:42.074498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3597fbd-5403-4219-875e-59ac23c3d4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:42.076080 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3597fbd-5403-4219-875e-59ac23c3d4f6 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:42.077331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:42.088294 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1376/2765] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:33:42.092007 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178aa07e-6044-4678-8289-d780f0213665 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:42.103383 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178aa07e-6044-4678-8289-d780f0213665 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:42.108277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa1d8b-acf0-4b59-8b97-fbb8ab6334a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:42.109028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa1d8b-acf0-4b59-8b97-fbb8ab6334a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:42.109485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa1d8b-acf0-4b59-8b97-fbb8ab6334a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:42.109766 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa1d8b-acf0-4b59-8b97-fbb8ab6334a5 admin admin] PATH_INFO: `/v3/projects/a98bccf3b93245dd950f6e1f1c924676/users/c4f4fd62f9b2498c9b5b5fc9de15f170/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:42.111729 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa1d8b-acf0-4b59-8b97-fbb8ab6334a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a98bccf3b93245dd950f6e1f1c924676, user_id=c4f4fd62f9b2498c9b5b5fc9de15f170, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:42.114625 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa1d8b-acf0-4b59-8b97-fbb8ab6334a5 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:42.129883 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1390/2766] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:33:42 2022] PUT /identity/v3/projects/a98bccf3b93245dd950f6e1f1c924676/users/c4f4fd62f9b2498c9b5b5fc9de15f170/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:42.876244 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41642e9a-34fe-44d7-b32b-02431519510c None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:42.876773 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41642e9a-34fe-44d7-b32b-02431519510c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:42.877019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41642e9a-34fe-44d7-b32b-02431519510c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:42.907835 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-41642e9a-34fe-44d7-b32b-02431519510c None None] MFA Rules not processed for user `c4f4fd62f9b2498c9b5b5fc9de15f170`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:42.909212 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-41642e9a-34fe-44d7-b32b-02431519510c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:42.990751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1377/2767] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:43.002082 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa1d8b-acf0-4b59-8b97-fbb8ab6334a5 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:43.003869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-ccfa1d8b-acf0-4b59-8b97-fbb8ab6334a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:43.062718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:43.071746 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa1d8b-acf0-4b59-8b97-fbb8ab6334a5 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:43.075497 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff1766-a74b-4be2-b298-5b8de272c490 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:43.076107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff1766-a74b-4be2-b298-5b8de272c490 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:43.076387 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff1766-a74b-4be2-b298-5b8de272c490 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:43.076693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff1766-a74b-4be2-b298-5b8de272c490 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:43.137006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff1766-a74b-4be2-b298-5b8de272c490 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:43.138402 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff1766-a74b-4be2-b298-5b8de272c490 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:43.147232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1391/2768] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:44.513904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41642e9a-34fe-44d7-b32b-02431519510c None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:44.524627 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41642e9a-34fe-44d7-b32b-02431519510c None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:44.528269 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538a5d3-c764-4113-aa00-c43b0c1592c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:44.528878 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538a5d3-c764-4113-aa00-c43b0c1592c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:44.529181 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538a5d3-c764-4113-aa00-c43b0c1592c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:44.529456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538a5d3-c764-4113-aa00-c43b0c1592c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:44.539337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538a5d3-c764-4113-aa00-c43b0c1592c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:44.540753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538a5d3-c764-4113-aa00-c43b0c1592c5 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:44.550046 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1378/2769] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:52.935973 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff1766-a74b-4be2-b298-5b8de272c490 service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:52.937430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-aaff1766-a74b-4be2-b298-5b8de272c490 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:52.991826 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff1766-a74b-4be2-b298-5b8de272c490 service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:52.996061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2a7b2-32df-462c-9f5d-3652eee82967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:52.996656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2a7b2-32df-462c-9f5d-3652eee82967 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:52.996930 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2a7b2-32df-462c-9f5d-3652eee82967 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:52.997230 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2a7b2-32df-462c-9f5d-3652eee82967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:52.998931 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-c2c2a7b2-32df-462c-9f5d-3652eee82967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:53.008160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2a7b2-32df-462c-9f5d-3652eee82967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:53.009591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2a7b2-32df-462c-9f5d-3652eee82967 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:53.017440 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1392/2770] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:33:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:33:54.545863 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=42 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:58.846630 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538a5d3-c764-4113-aa00-c43b0c1592c5 service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:58.910230 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538a5d3-c764-4113-aa00-c43b0c1592c5 service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:58.914617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861f267-58d0-4633-8cf4-3e8dd493c3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:58.915239 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861f267-58d0-4633-8cf4-3e8dd493c3eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:58.915517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861f267-58d0-4633-8cf4-3e8dd493c3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:58.915789 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861f267-58d0-4633-8cf4-3e8dd493c3eb admin admin] PATH_INFO: `/v3/users/508c3d45a6034ce088b77a622d4e4ef6` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:58.917220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861f267-58d0-4633-8cf4-3e8dd493c3eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=508c3d45a6034ce088b77a622d4e4ef6)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:58.919110 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861f267-58d0-4633-8cf4-3e8dd493c3eb admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:58.969333 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8861f267-58d0-4633-8cf4-3e8dd493c3eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '508c3d45a6034ce088b77a622d4e4ef6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:58.969981 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8861f267-58d0-4633-8cf4-3e8dd493c3eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '508c3d45a6034ce088b77a622d4e4ef6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:58.981646 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8861f267-58d0-4633-8cf4-3e8dd493c3eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '508c3d45a6034ce088b77a622d4e4ef6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:58.988534 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8861f267-58d0-4633-8cf4-3e8dd493c3eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '508c3d45a6034ce088b77a622d4e4ef6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:58.989185 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-8861f267-58d0-4633-8cf4-3e8dd493c3eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '508c3d45a6034ce088b77a622d4e4ef6'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:59.001418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1379/2771] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:33:58 2022] DELETE /identity/v3/users/508c3d45a6034ce088b77a622d4e4ef6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:59.177824 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2a7b2-32df-462c-9f5d-3652eee82967 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:59.192711 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2a7b2-32df-462c-9f5d-3652eee82967 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:59.254841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0ef58-e487-44d0-9566-8c7b55d2f27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:59.255481 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0ef58-e487-44d0-9566-8c7b55d2f27c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:59.255756 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0ef58-e487-44d0-9566-8c7b55d2f27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:59.256059 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0ef58-e487-44d0-9566-8c7b55d2f27c admin admin] PATH_INFO: `/v3/projects/4b6771e4c5eb46b6a0cd0ccc065e9b88` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:59.257268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba0ef58-e487-44d0-9566-8c7b55d2f27c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b6771e4c5eb46b6a0cd0ccc065e9b88)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:59.259208 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba0ef58-e487-44d0-9566-8c7b55d2f27c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:59.309961 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:59.311794 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-cba0ef58-e487-44d0-9566-8c7b55d2f27c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b6771e4c5eb46b6a0cd0ccc065e9b88', 'request_id': 'req-cba0ef58-e487-44d0-9566-8c7b55d2f27c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:59.321772 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-cba0ef58-e487-44d0-9566-8c7b55d2f27c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b6771e4c5eb46b6a0cd0ccc065e9b88', 'request_id': 'req-cba0ef58-e487-44d0-9566-8c7b55d2f27c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:59.329874 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-cba0ef58-e487-44d0-9566-8c7b55d2f27c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b6771e4c5eb46b6a0cd0ccc065e9b88', 'request_id': 'req-cba0ef58-e487-44d0-9566-8c7b55d2f27c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:59.342634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-cba0ef58-e487-44d0-9566-8c7b55d2f27c admin admin] The token cache is being invalidate because project 4b6771e4c5eb46b6a0cd0ccc065e9b88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:33:59.342634 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-cba0ef58-e487-44d0-9566-8c7b55d2f27c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:59.343287 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-cba0ef58-e487-44d0-9566-8c7b55d2f27c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:59.347630 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1393/2772] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:33:59 2022] DELETE /identity/v3/projects/4b6771e4c5eb46b6a0cd0ccc065e9b88 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:59.348886 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861f267-58d0-4633-8cf4-3e8dd493c3eb admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:59.350607 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8861f267-58d0-4633-8cf4-3e8dd493c3eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:59.364445 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861f267-58d0-4633-8cf4-3e8dd493c3eb admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:59.426464 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95c67d-911a-4ace-8896-0ac0ba13eb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:59.427031 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95c67d-911a-4ace-8896-0ac0ba13eb26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:59.427316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95c67d-911a-4ace-8896-0ac0ba13eb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:59.427587 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95c67d-911a-4ace-8896-0ac0ba13eb26 admin admin] PATH_INFO: `/v3/users/c4f4fd62f9b2498c9b5b5fc9de15f170` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:59.428778 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95c67d-911a-4ace-8896-0ac0ba13eb26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4f4fd62f9b2498c9b5b5fc9de15f170)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:59.430919 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95c67d-911a-4ace-8896-0ac0ba13eb26 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:59.454070 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:33:59.482467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ee95c67d-911a-4ace-8896-0ac0ba13eb26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4f4fd62f9b2498c9b5b5fc9de15f170'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:59.483520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ee95c67d-911a-4ace-8896-0ac0ba13eb26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4f4fd62f9b2498c9b5b5fc9de15f170'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:59.496594 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ee95c67d-911a-4ace-8896-0ac0ba13eb26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4f4fd62f9b2498c9b5b5fc9de15f170'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:59.504872 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ee95c67d-911a-4ace-8896-0ac0ba13eb26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4f4fd62f9b2498c9b5b5fc9de15f170'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:59.505656 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-ee95c67d-911a-4ace-8896-0ac0ba13eb26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4f4fd62f9b2498c9b5b5fc9de15f170'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:59.527843 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1380/2773] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:33:59 2022] DELETE /identity/v3/users/c4f4fd62f9b2498c9b5b5fc9de15f170 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:59.705544 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0ef58-e487-44d0-9566-8c7b55d2f27c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:33:59.707083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-cba0ef58-e487-44d0-9566-8c7b55d2f27c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:59.720024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0ef58-e487-44d0-9566-8c7b55d2f27c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:33:59.780882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d92ef0-e8ae-4a0b-aa99-d41b38698a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:33:59.781668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d92ef0-e8ae-4a0b-aa99-d41b38698a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:59.782175 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d92ef0-e8ae-4a0b-aa99-d41b38698a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:59.782609 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d92ef0-e8ae-4a0b-aa99-d41b38698a9d admin admin] PATH_INFO: `/v3/projects/a98bccf3b93245dd950f6e1f1c924676` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:59.784378 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d92ef0-e8ae-4a0b-aa99-d41b38698a9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a98bccf3b93245dd950f6e1f1c924676)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:33:59.786371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d92ef0-e8ae-4a0b-aa99-d41b38698a9d admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:33:59.835674 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b4d92ef0-e8ae-4a0b-aa99-d41b38698a9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a98bccf3b93245dd950f6e1f1c924676', 'request_id': 'req-b4d92ef0-e8ae-4a0b-aa99-d41b38698a9d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:59.843102 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b4d92ef0-e8ae-4a0b-aa99-d41b38698a9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a98bccf3b93245dd950f6e1f1c924676', 'request_id': 'req-b4d92ef0-e8ae-4a0b-aa99-d41b38698a9d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:59.851995 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b4d92ef0-e8ae-4a0b-aa99-d41b38698a9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a98bccf3b93245dd950f6e1f1c924676', 'request_id': 'req-b4d92ef0-e8ae-4a0b-aa99-d41b38698a9d'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:59.864038 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b4d92ef0-e8ae-4a0b-aa99-d41b38698a9d admin admin] The token cache is being invalidate because project a98bccf3b93245dd950f6e1f1c924676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:33:59.864327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b4d92ef0-e8ae-4a0b-aa99-d41b38698a9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:59.864991 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-b4d92ef0-e8ae-4a0b-aa99-d41b38698a9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:33:59.866498 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1394/2774] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:33:59 2022] DELETE /identity/v3/projects/a98bccf3b93245dd950f6e1f1c924676 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:33:59.871442 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9c482-40ef-4036-855a-8ca4a2477f6e None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:59.871993 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9c482-40ef-4036-855a-8ca4a2477f6e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:59.871993 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9c482-40ef-4036-855a-8ca4a2477f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:59.891767 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-84b9c482-40ef-4036-855a-8ca4a2477f6e None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:59.893021 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-84b9c482-40ef-4036-855a-8ca4a2477f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:59.956585 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1381/2775] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:33:59.964507 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bcde7-3dee-472e-beef-1fc4dfeae740 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:33:59.964797 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bcde7-3dee-472e-beef-1fc4dfeae740 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:33:59.965097 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bcde7-3dee-472e-beef-1fc4dfeae740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:33:59.986024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-343bcde7-3dee-472e-beef-1fc4dfeae740 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:33:59.987278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-343bcde7-3dee-472e-beef-1fc4dfeae740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:33:59.995560 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1395/2776] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:33:59.999934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9c482-40ef-4036-855a-8ca4a2477f6e None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:00.009687 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9c482-40ef-4036-855a-8ca4a2477f6e None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:00.014503 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bfc09-2152-4fa6-89b6-c1876e72fbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:00.015106 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bfc09-2152-4fa6-89b6-c1876e72fbbd admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:00.015388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bfc09-2152-4fa6-89b6-c1876e72fbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:00.015660 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bfc09-2152-4fa6-89b6-c1876e72fbbd admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:00.016316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bfc09-2152-4fa6-89b6-c1876e72fbbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:34:00.016616 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bfc09-2152-4fa6-89b6-c1876e72fbbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:00.018234 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bfc09-2152-4fa6-89b6-c1876e72fbbd admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:00.030510 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1382/2777] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:34:00.034887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bcde7-3dee-472e-beef-1fc4dfeae740 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:00.044945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bcde7-3dee-472e-beef-1fc4dfeae740 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:00.049668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4a951-9f89-4688-8a3b-fc883c1f11a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:00.050270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4a951-9f89-4688-8a3b-fc883c1f11a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:00.050547 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4a951-9f89-4688-8a3b-fc883c1f11a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:00.050815 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4a951-9f89-4688-8a3b-fc883c1f11a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:00.051446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4a951-9f89-4688-8a3b-fc883c1f11a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:00.053403 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf4a951-9f89-4688-8a3b-fc883c1f11a7 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:00.079922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1396/2778] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:34:00 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:34:00.084179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bfc09-2152-4fa6-89b6-c1876e72fbbd admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:00.095099 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bfc09-2152-4fa6-89b6-c1876e72fbbd admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:00.150430 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90853e-379e-4a4f-9d94-d366fa47b1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:00.151035 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90853e-379e-4a4f-9d94-d366fa47b1b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:00.151316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90853e-379e-4a4f-9d94-d366fa47b1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:00.151589 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90853e-379e-4a4f-9d94-d366fa47b1b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:00.152808 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90853e-379e-4a4f-9d94-d366fa47b1b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:00.154794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90853e-379e-4a4f-9d94-d366fa47b1b0 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:00.179961 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1383/2779] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:34:00 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:34:00.181021 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4a951-9f89-4688-8a3b-fc883c1f11a7 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:00.191346 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4a951-9f89-4688-8a3b-fc883c1f11a7 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:00.196114 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eed9a8-85dd-4449-8503-9ac33ce467e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:00.196717 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eed9a8-85dd-4449-8503-9ac33ce467e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:00.196998 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eed9a8-85dd-4449-8503-9ac33ce467e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:00.197331 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eed9a8-85dd-4449-8503-9ac33ce467e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:00.197982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eed9a8-85dd-4449-8503-9ac33ce467e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:00.200229 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eed9a8-85dd-4449-8503-9ac33ce467e8 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:00.210181 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1397/2780] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:34:00.213991 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90853e-379e-4a4f-9d94-d366fa47b1b0 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:00.224889 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90853e-379e-4a4f-9d94-d366fa47b1b0 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:00.231364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02646d-2d07-4983-ad21-f90b7d7ef7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:00.232029 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02646d-2d07-4983-ad21-f90b7d7ef7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:00.232382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02646d-2d07-4983-ad21-f90b7d7ef7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:00.232668 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02646d-2d07-4983-ad21-f90b7d7ef7d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:00.233384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02646d-2d07-4983-ad21-f90b7d7ef7d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:00.234984 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02646d-2d07-4983-ad21-f90b7d7ef7d7 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:00.246019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1384/2781] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:34:00.250085 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eed9a8-85dd-4449-8503-9ac33ce467e8 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:00.260327 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eed9a8-85dd-4449-8503-9ac33ce467e8 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:00.265665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d611eb8-92bc-4800-9611-0aa9324ab6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:00.266312 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d611eb8-92bc-4800-9611-0aa9324ab6e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:00.266590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d611eb8-92bc-4800-9611-0aa9324ab6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:00.266882 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d611eb8-92bc-4800-9611-0aa9324ab6e8 admin admin] PATH_INFO: `/v3/projects/61759cfe832147ee97afafcd4d236f30/users/68b1b5be361b4eef9e71f557150dd453/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:00.282050 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d611eb8-92bc-4800-9611-0aa9324ab6e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61759cfe832147ee97afafcd4d236f30, user_id=68b1b5be361b4eef9e71f557150dd453, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:00.284691 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d611eb8-92bc-4800-9611-0aa9324ab6e8 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:00.297250 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1398/2782] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:34:00 2022] PUT /identity/v3/projects/61759cfe832147ee97afafcd4d236f30/users/68b1b5be361b4eef9e71f557150dd453/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:34:00.307118 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76696d-9bdb-4021-bd54-2fb82b610d00 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:00.307404 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76696d-9bdb-4021-bd54-2fb82b610d00 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:00.307677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76696d-9bdb-4021-bd54-2fb82b610d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:00.315860 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:34:00.338479 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-0e76696d-9bdb-4021-bd54-2fb82b610d00 None None] MFA Rules not processed for user `68b1b5be361b4eef9e71f557150dd453`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:34:00.340537 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0e76696d-9bdb-4021-bd54-2fb82b610d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:00.406322 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1385/2783] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:34:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:34:00.410701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d611eb8-92bc-4800-9611-0aa9324ab6e8 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:00.415191 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:34:00.474402 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d611eb8-92bc-4800-9611-0aa9324ab6e8 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:00.479177 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf55c90-a171-490c-b2dd-f512745da313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:00.479785 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf55c90-a171-490c-b2dd-f512745da313 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:00.480282 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf55c90-a171-490c-b2dd-f512745da313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:00.480624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf55c90-a171-490c-b2dd-f512745da313 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:00.481309 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf55c90-a171-490c-b2dd-f512745da313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:00.483210 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf55c90-a171-490c-b2dd-f512745da313 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:00.509396 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1399/2784] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:34:00 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:34:00.514014 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76696d-9bdb-4021-bd54-2fb82b610d00 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:00.525114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76696d-9bdb-4021-bd54-2fb82b610d00 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:00.585689 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d9d46-cfc5-4974-80d2-ffcaa6198496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:00.586316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d9d46-cfc5-4974-80d2-ffcaa6198496 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:00.586599 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d9d46-cfc5-4974-80d2-ffcaa6198496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:00.586874 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d9d46-cfc5-4974-80d2-ffcaa6198496 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:00.587530 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d9d46-cfc5-4974-80d2-ffcaa6198496 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:00.589948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4d9d46-cfc5-4974-80d2-ffcaa6198496 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:00.612861 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1386/2785] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:34:00 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:34:00.616737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf55c90-a171-490c-b2dd-f512745da313 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:00.628156 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf55c90-a171-490c-b2dd-f512745da313 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:00.632830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c725e-b2d4-4cb1-bdd3-42dd484e8fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:00.633463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c725e-b2d4-4cb1-bdd3-42dd484e8fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:00.633759 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c725e-b2d4-4cb1-bdd3-42dd484e8fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:00.634020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c725e-b2d4-4cb1-bdd3-42dd484e8fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:00.634656 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c725e-b2d4-4cb1-bdd3-42dd484e8fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:00.636407 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c725e-b2d4-4cb1-bdd3-42dd484e8fe7 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:00.648040 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1400/2786] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:34:00.652349 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d9d46-cfc5-4974-80d2-ffcaa6198496 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:00.664026 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d9d46-cfc5-4974-80d2-ffcaa6198496 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:00.669749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f35a4-0ff1-45a2-bac9-e157ed9e18f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:00.670413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f35a4-0ff1-45a2-bac9-e157ed9e18f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:00.670696 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f35a4-0ff1-45a2-bac9-e157ed9e18f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:00.670988 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f35a4-0ff1-45a2-bac9-e157ed9e18f6 admin admin] PATH_INFO: `/v3/projects/7afdfb8774ad453dbfdbffe656d94424/users/cd7dd6003ffd472eb7edca80ebdb4efd/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:00.687096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f35a4-0ff1-45a2-bac9-e157ed9e18f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7afdfb8774ad453dbfdbffe656d94424, user_id=cd7dd6003ffd472eb7edca80ebdb4efd, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:00.689777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f35a4-0ff1-45a2-bac9-e157ed9e18f6 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:00.704653 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1387/2787] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:34:00 2022] PUT /identity/v3/projects/7afdfb8774ad453dbfdbffe656d94424/users/cd7dd6003ffd472eb7edca80ebdb4efd/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:34:00.708620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c725e-b2d4-4cb1-bdd3-42dd484e8fe7 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:00.719283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c725e-b2d4-4cb1-bdd3-42dd484e8fe7 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:00.774916 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfac244-6f86-4a9f-94a7-8fc028905df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:00.775444 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfac244-6f86-4a9f-94a7-8fc028905df9 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:00.775720 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfac244-6f86-4a9f-94a7-8fc028905df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:00.775991 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfac244-6f86-4a9f-94a7-8fc028905df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:00.776615 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfac244-6f86-4a9f-94a7-8fc028905df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:00.778212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfac244-6f86-4a9f-94a7-8fc028905df9 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:00.787841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1401/2788] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:34:00.791885 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f35a4-0ff1-45a2-bac9-e157ed9e18f6 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:00.802262 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f35a4-0ff1-45a2-bac9-e157ed9e18f6 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:00.806961 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edd30e-2d05-4345-82c4-a837caf2ea11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:00.807617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edd30e-2d05-4345-82c4-a837caf2ea11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:00.807899 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edd30e-2d05-4345-82c4-a837caf2ea11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:00.808761 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edd30e-2d05-4345-82c4-a837caf2ea11 admin admin] PATH_INFO: `/v3/projects/7afdfb8774ad453dbfdbffe656d94424/users/cd7dd6003ffd472eb7edca80ebdb4efd/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:00.810773 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51edd30e-2d05-4345-82c4-a837caf2ea11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7afdfb8774ad453dbfdbffe656d94424, user_id=cd7dd6003ffd472eb7edca80ebdb4efd, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:00.813384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51edd30e-2d05-4345-82c4-a837caf2ea11 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:00.823715 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1388/2789] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:34:00 2022] PUT /identity/v3/projects/7afdfb8774ad453dbfdbffe656d94424/users/cd7dd6003ffd472eb7edca80ebdb4efd/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:34:00.888694 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cf680-13ff-4e53-b5f0-0b3b1acc447b None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:00.888982 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cf680-13ff-4e53-b5f0-0b3b1acc447b None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:00.889302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cf680-13ff-4e53-b5f0-0b3b1acc447b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:00.916554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-e75cf680-13ff-4e53-b5f0-0b3b1acc447b None None] MFA Rules not processed for user `cd7dd6003ffd472eb7edca80ebdb4efd`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:34:00.918502 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e75cf680-13ff-4e53-b5f0-0b3b1acc447b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:00.983333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:34:00.990587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1402/2790] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:34:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:34:01.001378 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edd30e-2d05-4345-82c4-a837caf2ea11 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:01.003159 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-51edd30e-2d05-4345-82c4-a837caf2ea11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:01.055714 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:34:01.062276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edd30e-2d05-4345-82c4-a837caf2ea11 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:01.066283 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004d1ae-f545-4860-86ed-937e488a63a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:01.066880 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d1ae-f545-4860-86ed-937e488a63a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:01.067155 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d1ae-f545-4860-86ed-937e488a63a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:01.067425 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004d1ae-f545-4860-86ed-937e488a63a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:01.123937 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004d1ae-f545-4860-86ed-937e488a63a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:01.125405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004d1ae-f545-4860-86ed-937e488a63a0 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:01.134227 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1389/2791] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:34:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:34:04.064198 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb43e7-b5d2-4cd7-b971-0a926c7bfec7 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:04.064722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb43e7-b5d2-4cd7-b971-0a926c7bfec7 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:04.064722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb43e7-b5d2-4cd7-b971-0a926c7bfec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:04.084808 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-e6bb43e7-b5d2-4cd7-b971-0a926c7bfec7 None None] MFA Rules not processed for user `5b723fcdba1d4b26bd0cf604d59f8d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:34:04.086103 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e6bb43e7-b5d2-4cd7-b971-0a926c7bfec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:04.142705 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1403/2792] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:34:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:34:05.405514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004d1ae-f545-4860-86ed-937e488a63a0 service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:05.406905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c004d1ae-f545-4860-86ed-937e488a63a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:05.418708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004d1ae-f545-4860-86ed-937e488a63a0 service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:05.422287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85750a9-4829-4f98-b5e9-3638f022bdf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:05.422928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85750a9-4829-4f98-b5e9-3638f022bdf0 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:05.423155 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85750a9-4829-4f98-b5e9-3638f022bdf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:05.423426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85750a9-4829-4f98-b5e9-3638f022bdf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:05.432144 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85750a9-4829-4f98-b5e9-3638f022bdf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:05.434359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85750a9-4829-4f98-b5e9-3638f022bdf0 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:05.443625 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1390/2793] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:34:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:34:06.929232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb43e7-b5d2-4cd7-b971-0a926c7bfec7 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:06.930777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e6bb43e7-b5d2-4cd7-b971-0a926c7bfec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:07.018612 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb43e7-b5d2-4cd7-b971-0a926c7bfec7 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:07.024023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f232ece-d684-454e-ae69-d372d896c4ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:07.025212 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f232ece-d684-454e-ae69-d372d896c4ca service placement] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:07.025696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f232ece-d684-454e-ae69-d372d896c4ca service placement] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:07.026170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f232ece-d684-454e-ae69-d372d896c4ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:07.028140 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-0f232ece-d684-454e-ae69-d372d896c4ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:07.040962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f232ece-d684-454e-ae69-d372d896c4ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:07.042562 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f232ece-d684-454e-ae69-d372d896c4ca service placement] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:07.059551 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1404/2794] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:34:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:34:10.382094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85750a9-4829-4f98-b5e9-3638f022bdf0 service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:10.443325 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85750a9-4829-4f98-b5e9-3638f022bdf0 service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:10.447790 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23634ebc-59d9-4d18-a537-3b56f8ffbe1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:10.448405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23634ebc-59d9-4d18-a537-3b56f8ffbe1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:10.448681 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23634ebc-59d9-4d18-a537-3b56f8ffbe1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:10.448971 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23634ebc-59d9-4d18-a537-3b56f8ffbe1d admin admin] PATH_INFO: `/v3/users/68b1b5be361b4eef9e71f557150dd453` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:10.450755 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23634ebc-59d9-4d18-a537-3b56f8ffbe1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=68b1b5be361b4eef9e71f557150dd453)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:10.452691 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23634ebc-59d9-4d18-a537-3b56f8ffbe1d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:10.505320 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-23634ebc-59d9-4d18-a537-3b56f8ffbe1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68b1b5be361b4eef9e71f557150dd453'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:10.506549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-23634ebc-59d9-4d18-a537-3b56f8ffbe1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68b1b5be361b4eef9e71f557150dd453'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:10.520236 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-23634ebc-59d9-4d18-a537-3b56f8ffbe1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68b1b5be361b4eef9e71f557150dd453'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:10.527584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-23634ebc-59d9-4d18-a537-3b56f8ffbe1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68b1b5be361b4eef9e71f557150dd453'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:10.528379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-23634ebc-59d9-4d18-a537-3b56f8ffbe1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68b1b5be361b4eef9e71f557150dd453'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:10.541366 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1391/2795] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:34:10 2022] DELETE /identity/v3/users/68b1b5be361b4eef9e71f557150dd453 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:34:10.545952 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9affed-8772-4fc6-bd5c-0a89d6073fb6 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:10.546464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9affed-8772-4fc6-bd5c-0a89d6073fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:10.546581 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9affed-8772-4fc6-bd5c-0a89d6073fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:10.566379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-fe9affed-8772-4fc6-bd5c-0a89d6073fb6 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:34:10.567814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-fe9affed-8772-4fc6-bd5c-0a89d6073fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:10.628693 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1405/2796] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:34:10.640122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23634ebc-59d9-4d18-a537-3b56f8ffbe1d admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:10.641572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-23634ebc-59d9-4d18-a537-3b56f8ffbe1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:10.651641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:34:10.704155 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23634ebc-59d9-4d18-a537-3b56f8ffbe1d admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:10.708240 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf694064-08a2-4c70-ae46-fbb5b46ce773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:10.708848 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf694064-08a2-4c70-ae46-fbb5b46ce773 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:10.709123 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf694064-08a2-4c70-ae46-fbb5b46ce773 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:10.709421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf694064-08a2-4c70-ae46-fbb5b46ce773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:10.719235 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf694064-08a2-4c70-ae46-fbb5b46ce773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:10.720720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf694064-08a2-4c70-ae46-fbb5b46ce773 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:10.731680 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1392/2797] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:34:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:34:11.000977 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9affed-8772-4fc6-bd5c-0a89d6073fb6 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:11.002510 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-fe9affed-8772-4fc6-bd5c-0a89d6073fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:11.016618 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9affed-8772-4fc6-bd5c-0a89d6073fb6 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:11.021337 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:11.022128 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:11.022415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:11.022689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1 admin admin] PATH_INFO: `/v3/projects/61759cfe832147ee97afafcd4d236f30` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:11.023820 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61759cfe832147ee97afafcd4d236f30)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:11.025759 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:11.058713 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:34:11.075954 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61759cfe832147ee97afafcd4d236f30', 'request_id': 'req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:11.082104 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61759cfe832147ee97afafcd4d236f30', 'request_id': 'req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:11.089873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61759cfe832147ee97afafcd4d236f30', 'request_id': 'req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:11.101753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1 admin admin] The token cache is being invalidate because project 61759cfe832147ee97afafcd4d236f30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:34:11.102045 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:11.102697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:11.104171 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1406/2798] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:34:10 2022] DELETE /identity/v3/projects/61759cfe832147ee97afafcd4d236f30 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:34:11.108551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf694064-08a2-4c70-ae46-fbb5b46ce773 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:11.118107 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf694064-08a2-4c70-ae46-fbb5b46ce773 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:11.183752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71bf987-3e8a-4154-bd4c-1d8c7d3710cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:11.185116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bf987-3e8a-4154-bd4c-1d8c7d3710cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:11.185521 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bf987-3e8a-4154-bd4c-1d8c7d3710cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:11.185970 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bf987-3e8a-4154-bd4c-1d8c7d3710cf admin admin] PATH_INFO: `/v3/users/cd7dd6003ffd472eb7edca80ebdb4efd` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:11.187096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71bf987-3e8a-4154-bd4c-1d8c7d3710cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd7dd6003ffd472eb7edca80ebdb4efd)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:11.189018 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71bf987-3e8a-4154-bd4c-1d8c7d3710cf admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:11.238179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f71bf987-3e8a-4154-bd4c-1d8c7d3710cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd7dd6003ffd472eb7edca80ebdb4efd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:11.238967 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f71bf987-3e8a-4154-bd4c-1d8c7d3710cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd7dd6003ffd472eb7edca80ebdb4efd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:11.252345 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f71bf987-3e8a-4154-bd4c-1d8c7d3710cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd7dd6003ffd472eb7edca80ebdb4efd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:11.262708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f71bf987-3e8a-4154-bd4c-1d8c7d3710cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd7dd6003ffd472eb7edca80ebdb4efd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:11.263595 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-f71bf987-3e8a-4154-bd4c-1d8c7d3710cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd7dd6003ffd472eb7edca80ebdb4efd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:11.280705 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1393/2799] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:34:11 2022] DELETE /identity/v3/users/cd7dd6003ffd472eb7edca80ebdb4efd => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:34:11.608268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:11.610075 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:11.633957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a8dba-1e4b-4d02-bbb1-3d5b2cc50bf1 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:11.693018 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5afb8-14c9-42ee-9198-2278462319bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:11.693715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5afb8-14c9-42ee-9198-2278462319bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:11.693997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5afb8-14c9-42ee-9198-2278462319bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:11.694271 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5afb8-14c9-42ee-9198-2278462319bd admin admin] PATH_INFO: `/v3/projects/7afdfb8774ad453dbfdbffe656d94424` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:11.695390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5afb8-14c9-42ee-9198-2278462319bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7afdfb8774ad453dbfdbffe656d94424)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:11.697331 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5afb8-14c9-42ee-9198-2278462319bd admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:11.753810 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-22f5afb8-14c9-42ee-9198-2278462319bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7afdfb8774ad453dbfdbffe656d94424', 'request_id': 'req-22f5afb8-14c9-42ee-9198-2278462319bd'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:11.760600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-22f5afb8-14c9-42ee-9198-2278462319bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7afdfb8774ad453dbfdbffe656d94424', 'request_id': 'req-22f5afb8-14c9-42ee-9198-2278462319bd'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:11.768922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-22f5afb8-14c9-42ee-9198-2278462319bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7afdfb8774ad453dbfdbffe656d94424', 'request_id': 'req-22f5afb8-14c9-42ee-9198-2278462319bd'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:11.780123 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-22f5afb8-14c9-42ee-9198-2278462319bd admin admin] The token cache is being invalidate because project 7afdfb8774ad453dbfdbffe656d94424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:34:11.780408 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-22f5afb8-14c9-42ee-9198-2278462319bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:11.781043 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-22f5afb8-14c9-42ee-9198-2278462319bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:11.782561 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1407/2800] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:34:11 2022] DELETE /identity/v3/projects/7afdfb8774ad453dbfdbffe656d94424 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:34:11.789094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9eaa0c-635f-4579-a6f9-6f79e598ff01 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:11.789615 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9eaa0c-635f-4579-a6f9-6f79e598ff01 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:11.789708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9eaa0c-635f-4579-a6f9-6f79e598ff01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:11.810098 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-fb9eaa0c-635f-4579-a6f9-6f79e598ff01 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:34:11.811303 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-fb9eaa0c-635f-4579-a6f9-6f79e598ff01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:11.835763 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:34:11.875651 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1394/2801] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:34:11.884251 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48afcb6-4a10-44c5-b970-9b0528689774 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:11.884539 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48afcb6-4a10-44c5-b970-9b0528689774 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:11.884838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48afcb6-4a10-44c5-b970-9b0528689774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:11.906430 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-b48afcb6-4a10-44c5-b970-9b0528689774 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:34:11.907623 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b48afcb6-4a10-44c5-b970-9b0528689774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:11.915956 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1408/2802] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:34:11.920670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9eaa0c-635f-4579-a6f9-6f79e598ff01 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:11.931070 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9eaa0c-635f-4579-a6f9-6f79e598ff01 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:11.936687 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c95494-8856-4365-8f81-fc22d96f4434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:11.937329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c95494-8856-4365-8f81-fc22d96f4434 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:11.937608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c95494-8856-4365-8f81-fc22d96f4434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:11.937877 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c95494-8856-4365-8f81-fc22d96f4434 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:11.938541 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c95494-8856-4365-8f81-fc22d96f4434 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:34:11.938823 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c95494-8856-4365-8f81-fc22d96f4434 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:11.940945 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c95494-8856-4365-8f81-fc22d96f4434 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:11.954689 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1395/2803] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:34:11.960022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48afcb6-4a10-44c5-b970-9b0528689774 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:11.973912 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48afcb6-4a10-44c5-b970-9b0528689774 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:11.980378 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0abb3-bea9-4621-92d6-77ce8f8c463a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:11.981590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0abb3-bea9-4621-92d6-77ce8f8c463a admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:11.981884 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0abb3-bea9-4621-92d6-77ce8f8c463a admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:11.982219 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0abb3-bea9-4621-92d6-77ce8f8c463a admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:11.982869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d0abb3-bea9-4621-92d6-77ce8f8c463a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:11.984926 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d0abb3-bea9-4621-92d6-77ce8f8c463a admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:12.025758 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1409/2804] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:34:11 2022] POST /identity/v3/projects => generated 367 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:34:12.034687 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c95494-8856-4365-8f81-fc22d96f4434 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:12.045975 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c95494-8856-4365-8f81-fc22d96f4434 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:12.124774 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973133be-c3cd-4e7b-b263-5e554bf2bb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:12.125427 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973133be-c3cd-4e7b-b263-5e554bf2bb1a admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:12.125708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973133be-c3cd-4e7b-b263-5e554bf2bb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:12.125979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973133be-c3cd-4e7b-b263-5e554bf2bb1a admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:12.126613 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973133be-c3cd-4e7b-b263-5e554bf2bb1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:12.129177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973133be-c3cd-4e7b-b263-5e554bf2bb1a admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:12.163656 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1396/2805] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:34:12 2022] POST /identity/v3/users => generated 284 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:34:12.166417 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0abb3-bea9-4621-92d6-77ce8f8c463a admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:12.182057 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0abb3-bea9-4621-92d6-77ce8f8c463a admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:12.192612 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd7820-8c06-4251-b432-84a4a663d017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:12.194600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd7820-8c06-4251-b432-84a4a663d017 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:12.195481 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd7820-8c06-4251-b432-84a4a663d017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:12.195901 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd7820-8c06-4251-b432-84a4a663d017 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:12.196592 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd7820-8c06-4251-b432-84a4a663d017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:12.199022 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd7820-8c06-4251-b432-84a4a663d017 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:12.212374 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1410/2806] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:34:12.216399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973133be-c3cd-4e7b-b263-5e554bf2bb1a admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:12.226807 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973133be-c3cd-4e7b-b263-5e554bf2bb1a admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:12.231980 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027fafd0-7f7d-4f78-8e5b-d00541f9c875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:12.232604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fafd0-7f7d-4f78-8e5b-d00541f9c875 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:12.232887 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fafd0-7f7d-4f78-8e5b-d00541f9c875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:12.233184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027fafd0-7f7d-4f78-8e5b-d00541f9c875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:12.233822 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fafd0-7f7d-4f78-8e5b-d00541f9c875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:12.235411 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027fafd0-7f7d-4f78-8e5b-d00541f9c875 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:12.368725 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1397/2807] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:34:12.373672 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd7820-8c06-4251-b432-84a4a663d017 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:12.383415 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd7820-8c06-4251-b432-84a4a663d017 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:12.388033 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94994f-5f35-4e44-8997-ba310b93db52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:12.388679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94994f-5f35-4e44-8997-ba310b93db52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:12.388958 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94994f-5f35-4e44-8997-ba310b93db52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:12.389263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94994f-5f35-4e44-8997-ba310b93db52 admin admin] PATH_INFO: `/v3/projects/7f461c657067484f9e8b4639cae0d078/users/9171f04498a948beb97b1e890d7d16a4/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:12.391670 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:34:12.406917 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94994f-5f35-4e44-8997-ba310b93db52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f461c657067484f9e8b4639cae0d078, user_id=9171f04498a948beb97b1e890d7d16a4, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:12.409739 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94994f-5f35-4e44-8997-ba310b93db52 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:12.424154 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1411/2808] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:34:12 2022] PUT /identity/v3/projects/7f461c657067484f9e8b4639cae0d078/users/9171f04498a948beb97b1e890d7d16a4/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:34:12.436884 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd69997-37fa-42e4-969a-2ac2debd3f05 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:12.437191 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd69997-37fa-42e4-969a-2ac2debd3f05 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:12.437467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd69997-37fa-42e4-969a-2ac2debd3f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:12.464306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-2dd69997-37fa-42e4-969a-2ac2debd3f05 None None] MFA Rules not processed for user `9171f04498a948beb97b1e890d7d16a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:34:12.465600 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2dd69997-37fa-42e4-969a-2ac2debd3f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:12.542323 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1398/2809] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:34:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:34:12.551850 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94994f-5f35-4e44-8997-ba310b93db52 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:12.553591 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5d94994f-5f35-4e44-8997-ba310b93db52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:12.618574 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94994f-5f35-4e44-8997-ba310b93db52 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:12.622113 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f2634-d203-4ede-b22d-d4fb680b7903 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e08e00bcb5f47ffab58974ba376938a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:12.622878 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f2634-d203-4ede-b22d-d4fb680b7903 service glance] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:12.623758 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f2634-d203-4ede-b22d-d4fb680b7903 service glance] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:12.623758 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f2634-d203-4ede-b22d-d4fb680b7903 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:12.632659 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f2634-d203-4ede-b22d-d4fb680b7903 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:12.634187 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f2634-d203-4ede-b22d-d4fb680b7903 service glance] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:12.643745 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1412/2810] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:34:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:34:12.886248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd69997-37fa-42e4-969a-2ac2debd3f05 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:12.891028 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2dd69997-37fa-42e4-969a-2ac2debd3f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:12.949213 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:34:13.029033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd69997-37fa-42e4-969a-2ac2debd3f05 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:13.036140 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbd434-7d3a-466a-8ca9-e5a02ad38125 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:13.036858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbd434-7d3a-466a-8ca9-e5a02ad38125 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:13.037218 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbd434-7d3a-466a-8ca9-e5a02ad38125 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:13.037551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbd434-7d3a-466a-8ca9-e5a02ad38125 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:13.039396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e2fbd434-7d3a-466a-8ca9-e5a02ad38125 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:13.053406 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbd434-7d3a-466a-8ca9-e5a02ad38125 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:13.055116 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbd434-7d3a-466a-8ca9-e5a02ad38125 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:13.066772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1399/2811] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:34:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:34:22.639741 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:34:23.061611 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=23 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:34:24.170273 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f2634-d203-4ede-b22d-d4fb680b7903 service glance] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:24.183249 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f2634-d203-4ede-b22d-d4fb680b7903 service glance] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:24.188734 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd85bd49-0319-4c1c-8dc8-67d69554a061 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:24.189381 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85bd49-0319-4c1c-8dc8-67d69554a061 service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:24.189666 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85bd49-0319-4c1c-8dc8-67d69554a061 service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:24.189939 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85bd49-0319-4c1c-8dc8-67d69554a061 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:24.191758 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-cd85bd49-0319-4c1c-8dc8-67d69554a061 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:24.204905 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd85bd49-0319-4c1c-8dc8-67d69554a061 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:24.206528 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd85bd49-0319-4c1c-8dc8-67d69554a061 service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:24.218166 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1413/2812] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:34:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:34:27.083416 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbd434-7d3a-466a-8ca9-e5a02ad38125 service nova] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:27.084934 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e2fbd434-7d3a-466a-8ca9-e5a02ad38125 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:27.139968 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbd434-7d3a-466a-8ca9-e5a02ad38125 service nova] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:27.143737 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fc8df-d48b-4b99-a074-c3f44f3d4a51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:27.144328 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fc8df-d48b-4b99-a074-c3f44f3d4a51 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:27.144601 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fc8df-d48b-4b99-a074-c3f44f3d4a51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:27.144871 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32fc8df-d48b-4b99-a074-c3f44f3d4a51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:27.146559 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e32fc8df-d48b-4b99-a074-c3f44f3d4a51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:27.207739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32fc8df-d48b-4b99-a074-c3f44f3d4a51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:27.209154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32fc8df-d48b-4b99-a074-c3f44f3d4a51 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:27.218754 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1400/2813] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:34:27 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:34:28.997578 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd85bd49-0319-4c1c-8dc8-67d69554a061 service nova] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:28.999246 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-cd85bd49-0319-4c1c-8dc8-67d69554a061 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:29.013372 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd85bd49-0319-4c1c-8dc8-67d69554a061 service nova] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:29.018840 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c39c80-22b6-4543-a348-ac20bef57c8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:29.019439 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c39c80-22b6-4543-a348-ac20bef57c8f service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:29.019718 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c39c80-22b6-4543-a348-ac20bef57c8f service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:29.019991 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c39c80-22b6-4543-a348-ac20bef57c8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:29.021750 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-04c39c80-22b6-4543-a348-ac20bef57c8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:29.034586 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c39c80-22b6-4543-a348-ac20bef57c8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:29.036138 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c39c80-22b6-4543-a348-ac20bef57c8f service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:29.047400 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1414/2814] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:34:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:34:37.214216 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=50 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:34:39.042114 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=12 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:34:57.860865 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fc8df-d48b-4b99-a074-c3f44f3d4a51 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:57.920896 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32fc8df-d48b-4b99-a074-c3f44f3d4a51 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:57.925413 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fb441-4963-4d0e-9423-a41f0a041d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:57.926024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fb441-4963-4d0e-9423-a41f0a041d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:57.926301 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fb441-4963-4d0e-9423-a41f0a041d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:57.926572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fb441-4963-4d0e-9423-a41f0a041d62 admin admin] PATH_INFO: `/v3/users/9171f04498a948beb97b1e890d7d16a4` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:57.927667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fb441-4963-4d0e-9423-a41f0a041d62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9171f04498a948beb97b1e890d7d16a4)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:57.929588 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652fb441-4963-4d0e-9423-a41f0a041d62 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:57.975306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-652fb441-4963-4d0e-9423-a41f0a041d62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9171f04498a948beb97b1e890d7d16a4'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:57.975947 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-652fb441-4963-4d0e-9423-a41f0a041d62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9171f04498a948beb97b1e890d7d16a4'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:57.988501 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-652fb441-4963-4d0e-9423-a41f0a041d62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9171f04498a948beb97b1e890d7d16a4'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:57.995769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-652fb441-4963-4d0e-9423-a41f0a041d62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9171f04498a948beb97b1e890d7d16a4'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:57.996410 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-652fb441-4963-4d0e-9423-a41f0a041d62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9171f04498a948beb97b1e890d7d16a4'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:58.009216 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1401/2815] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:34:57 2022] DELETE /identity/v3/users/9171f04498a948beb97b1e890d7d16a4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:34:58.013736 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db453127-791e-42d6-861f-0a737db5a126 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:58.014227 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db453127-791e-42d6-861f-0a737db5a126 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:58.014348 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db453127-791e-42d6-861f-0a737db5a126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:58.033837 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-db453127-791e-42d6-861f-0a737db5a126 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:34:58.035023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-db453127-791e-42d6-861f-0a737db5a126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:58.092432 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1415/2816] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:34:58.102017 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fb441-4963-4d0e-9423-a41f0a041d62 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:58.103395 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-652fb441-4963-4d0e-9423-a41f0a041d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:58.160420 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fb441-4963-4d0e-9423-a41f0a041d62 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:58.164067 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c0241-7621-4ddb-be1b-261fafd49173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:58.164660 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c0241-7621-4ddb-be1b-261fafd49173 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:58.164942 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c0241-7621-4ddb-be1b-261fafd49173 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:58.165243 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c0241-7621-4ddb-be1b-261fafd49173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:58.174902 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c0241-7621-4ddb-be1b-261fafd49173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:58.176429 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c0241-7621-4ddb-be1b-261fafd49173 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:58.187197 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1402/2817] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:34:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:34:58.727709 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db453127-791e-42d6-861f-0a737db5a126 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:58.729160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-db453127-791e-42d6-861f-0a737db5a126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:58.742537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db453127-791e-42d6-861f-0a737db5a126 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:58.747592 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc1992-ce09-4dcd-930d-c01ff1574890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:58.749340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc1992-ce09-4dcd-930d-c01ff1574890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:58.749812 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc1992-ce09-4dcd-930d-c01ff1574890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:58.750319 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc1992-ce09-4dcd-930d-c01ff1574890 admin admin] PATH_INFO: `/v3/projects/7f461c657067484f9e8b4639cae0d078` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:58.752401 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc1992-ce09-4dcd-930d-c01ff1574890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f461c657067484f9e8b4639cae0d078)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:58.756194 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc1992-ce09-4dcd-930d-c01ff1574890 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:58.814816 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-13cc1992-ce09-4dcd-930d-c01ff1574890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f461c657067484f9e8b4639cae0d078', 'request_id': 'req-13cc1992-ce09-4dcd-930d-c01ff1574890'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:58.821663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-13cc1992-ce09-4dcd-930d-c01ff1574890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f461c657067484f9e8b4639cae0d078', 'request_id': 'req-13cc1992-ce09-4dcd-930d-c01ff1574890'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:58.833476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-13cc1992-ce09-4dcd-930d-c01ff1574890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f461c657067484f9e8b4639cae0d078', 'request_id': 'req-13cc1992-ce09-4dcd-930d-c01ff1574890'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:58.844828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-13cc1992-ce09-4dcd-930d-c01ff1574890 admin admin] The token cache is being invalidate because project 7f461c657067484f9e8b4639cae0d078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:34:58.845113 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-13cc1992-ce09-4dcd-930d-c01ff1574890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:58.845830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-13cc1992-ce09-4dcd-930d-c01ff1574890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:34:58.847345 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1416/2818] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:34:58 2022] DELETE /identity/v3/projects/7f461c657067484f9e8b4639cae0d078 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:34:58.852315 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a02bd-07bf-4aad-bdeb-b39bed8a0fcc None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:58.852629 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a02bd-07bf-4aad-bdeb-b39bed8a0fcc None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:58.852900 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a02bd-07bf-4aad-bdeb-b39bed8a0fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:58.874369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-bd4a02bd-07bf-4aad-bdeb-b39bed8a0fcc None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:34:58.875566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-bd4a02bd-07bf-4aad-bdeb-b39bed8a0fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:58.894222 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:34:58.948753 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1403/2819] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:34:58.957638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e904bf-868d-44d2-b7d2-cae29f5fb7cb None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:58.958026 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e904bf-868d-44d2-b7d2-cae29f5fb7cb None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:58.958329 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e904bf-868d-44d2-b7d2-cae29f5fb7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:58.979800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-32e904bf-868d-44d2-b7d2-cae29f5fb7cb None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:34:58.980931 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-32e904bf-868d-44d2-b7d2-cae29f5fb7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:58.989886 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1417/2820] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:34:58.994567 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a02bd-07bf-4aad-bdeb-b39bed8a0fcc None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:59.006341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a02bd-07bf-4aad-bdeb-b39bed8a0fcc None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:59.011072 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affbff55-d11c-4754-9013-c53156bfc664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:59.011671 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbff55-d11c-4754-9013-c53156bfc664 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:59.011946 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbff55-d11c-4754-9013-c53156bfc664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:59.012242 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbff55-d11c-4754-9013-c53156bfc664 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:59.013312 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbff55-d11c-4754-9013-c53156bfc664 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:34:59.013711 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbff55-d11c-4754-9013-c53156bfc664 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:59.015290 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbff55-d11c-4754-9013-c53156bfc664 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:59.026989 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1404/2821] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:34:59.032360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e904bf-868d-44d2-b7d2-cae29f5fb7cb None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:59.042333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e904bf-868d-44d2-b7d2-cae29f5fb7cb None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:59.047719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3c45d-24d6-4c61-80d7-ba232f54d030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:59.048351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3c45d-24d6-4c61-80d7-ba232f54d030 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:59.048631 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3c45d-24d6-4c61-80d7-ba232f54d030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:59.048904 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3c45d-24d6-4c61-80d7-ba232f54d030 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:59.049573 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3c45d-24d6-4c61-80d7-ba232f54d030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:59.051455 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3c45d-24d6-4c61-80d7-ba232f54d030 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:59.079775 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1418/2822] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:34:59 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:34:59.084203 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affbff55-d11c-4754-9013-c53156bfc664 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:59.097447 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affbff55-d11c-4754-9013-c53156bfc664 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:59.158670 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df7f87-a47c-487e-98b0-88d8c534f480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:59.159511 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df7f87-a47c-487e-98b0-88d8c534f480 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:59.159788 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df7f87-a47c-487e-98b0-88d8c534f480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:59.160058 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df7f87-a47c-487e-98b0-88d8c534f480 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:59.160712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df7f87-a47c-487e-98b0-88d8c534f480 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:59.162609 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df7f87-a47c-487e-98b0-88d8c534f480 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:59.186093 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1405/2823] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:34:59 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:34:59.190355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3c45d-24d6-4c61-80d7-ba232f54d030 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:59.200991 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3c45d-24d6-4c61-80d7-ba232f54d030 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:59.206403 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ce744-e46c-4f62-a06d-df735d51a290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:59.207451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ce744-e46c-4f62-a06d-df735d51a290 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:59.207962 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ce744-e46c-4f62-a06d-df735d51a290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:59.208461 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ce744-e46c-4f62-a06d-df735d51a290 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:59.209668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ce744-e46c-4f62-a06d-df735d51a290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:59.212657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ce744-e46c-4f62-a06d-df735d51a290 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:59.224158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1419/2824] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:34:59.228614 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df7f87-a47c-487e-98b0-88d8c534f480 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:59.242212 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df7f87-a47c-487e-98b0-88d8c534f480 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:59.246899 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c3553-f9a3-4596-9357-0117e813482a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:59.247509 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c3553-f9a3-4596-9357-0117e813482a admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:59.248120 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c3553-f9a3-4596-9357-0117e813482a admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:59.248616 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c3553-f9a3-4596-9357-0117e813482a admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:59.249397 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c3553-f9a3-4596-9357-0117e813482a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:59.250997 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c3553-f9a3-4596-9357-0117e813482a admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:59.267995 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1406/2825] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:34:59.269355 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ce744-e46c-4f62-a06d-df735d51a290 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:59.280356 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ce744-e46c-4f62-a06d-df735d51a290 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:59.285485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a9a1c-0e58-40d6-9ca0-61b3c42937de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:59.286152 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a9a1c-0e58-40d6-9ca0-61b3c42937de admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:59.286432 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a9a1c-0e58-40d6-9ca0-61b3c42937de admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:59.286708 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a9a1c-0e58-40d6-9ca0-61b3c42937de admin admin] PATH_INFO: `/v3/projects/e6eccd42ceef4259b48149c9835fc73b/users/42450ebae9bb445dba460ea02832a173/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:59.302454 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a9a1c-0e58-40d6-9ca0-61b3c42937de admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6eccd42ceef4259b48149c9835fc73b, user_id=42450ebae9bb445dba460ea02832a173, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:59.305119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a9a1c-0e58-40d6-9ca0-61b3c42937de admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:59.318478 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1420/2826] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:34:59 2022] PUT /identity/v3/projects/e6eccd42ceef4259b48149c9835fc73b/users/42450ebae9bb445dba460ea02832a173/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:34:59.323948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a8a17-7cb7-4c0a-b018-3bafe0477e13 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:59.324237 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a8a17-7cb7-4c0a-b018-3bafe0477e13 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:59.324510 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a8a17-7cb7-4c0a-b018-3bafe0477e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:59.344961 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-de5a8a17-7cb7-4c0a-b018-3bafe0477e13 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:34:59.346193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-de5a8a17-7cb7-4c0a-b018-3bafe0477e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:59.407474 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1407/2827] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:34:59.421657 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a9a1c-0e58-40d6-9ca0-61b3c42937de admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:34:59.429485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-8d4a9a1c-0e58-40d6-9ca0-61b3c42937de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:34:59.458363 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:34:59.511588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a9a1c-0e58-40d6-9ca0-61b3c42937de admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:34:59.517777 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb041c-5ecb-435e-b9b2-b0a8febe2bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:34:59.518391 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb041c-5ecb-435e-b9b2-b0a8febe2bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:34:59.518676 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb041c-5ecb-435e-b9b2-b0a8febe2bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:34:59.518948 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb041c-5ecb-435e-b9b2-b0a8febe2bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:34:59.530635 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb041c-5ecb-435e-b9b2-b0a8febe2bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:34:59.532058 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb041c-5ecb-435e-b9b2-b0a8febe2bb8 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:34:59.550393 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1421/2828] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:34:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:35:00.183667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3d60a-58b0-470e-8e1f-4b3760683251 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:35:00.184183 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3d60a-58b0-470e-8e1f-4b3760683251 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:35:00.184285 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3d60a-58b0-470e-8e1f-4b3760683251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:35:00.213112 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-7fe3d60a-58b0-470e-8e1f-4b3760683251 None None] MFA Rules not processed for user `42450ebae9bb445dba460ea02832a173`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:35:00.214434 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-7fe3d60a-58b0-470e-8e1f-4b3760683251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:35:00.233700 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:35:00.284898 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1408/2829] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:35:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:35:00.297250 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb041c-5ecb-435e-b9b2-b0a8febe2bb8 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:35:00.299830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-19eb041c-5ecb-435e-b9b2-b0a8febe2bb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:35:00.357122 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb041c-5ecb-435e-b9b2-b0a8febe2bb8 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:35:00.361060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd509f-2f96-4c75-98e3-ca84595cae0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:35:00.361711 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd509f-2f96-4c75-98e3-ca84595cae0d service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:35:00.361987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd509f-2f96-4c75-98e3-ca84595cae0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:35:00.362309 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd509f-2f96-4c75-98e3-ca84595cae0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:35:00.371186 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd509f-2f96-4c75-98e3-ca84595cae0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:35:00.372910 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd509f-2f96-4c75-98e3-ca84595cae0d service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:35:00.383242 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1422/2830] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:35:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:35:01.741254 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7853a-ad32-4a1b-8619-905f01560f58 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:35:01.741784 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7853a-ad32-4a1b-8619-905f01560f58 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:35:01.741784 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7853a-ad32-4a1b-8619-905f01560f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:35:01.761464 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-dbe7853a-ad32-4a1b-8619-905f01560f58 None None] MFA Rules not processed for user `42450ebae9bb445dba460ea02832a173`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:35:01.762660 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-dbe7853a-ad32-4a1b-8619-905f01560f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:35:01.769660 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1409/2831] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:35:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:35:02.001061 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd509f-2f96-4c75-98e3-ca84595cae0d service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:35:02.002464 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-81fd509f-2f96-4c75-98e3-ca84595cae0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:35:02.014776 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd509f-2f96-4c75-98e3-ca84595cae0d service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:35:02.018398 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f2e08-1e84-4379-8d65-a08a8fc35883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:35:02.018984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f2e08-1e84-4379-8d65-a08a8fc35883 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:35:02.019261 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f2e08-1e84-4379-8d65-a08a8fc35883 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:35:02.019530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f2e08-1e84-4379-8d65-a08a8fc35883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:35:02.021159 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-168f2e08-1e84-4379-8d65-a08a8fc35883 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:35:02.030184 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f2e08-1e84-4379-8d65-a08a8fc35883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:35:02.031662 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f2e08-1e84-4379-8d65-a08a8fc35883 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:35:02.039427 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1423/2832] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:35:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:35:11.756091 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=22 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:35:12.035887 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=49 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:36:08.084063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7853a-ad32-4a1b-8619-905f01560f58 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:36:08.085602 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-dbe7853a-ad32-4a1b-8619-905f01560f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:36:08.153777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7853a-ad32-4a1b-8619-905f01560f58 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:36:08.159414 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07af44-f4ef-4dda-a09b-52a2bf88a4a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:36:08.160021 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07af44-f4ef-4dda-a09b-52a2bf88a4a8 service placement] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:36:08.160302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07af44-f4ef-4dda-a09b-52a2bf88a4a8 service placement] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:36:08.160572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07af44-f4ef-4dda-a09b-52a2bf88a4a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:36:08.162360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-5d07af44-f4ef-4dda-a09b-52a2bf88a4a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:36:08.174862 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07af44-f4ef-4dda-a09b-52a2bf88a4a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:36:08.176441 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07af44-f4ef-4dda-a09b-52a2bf88a4a8 service placement] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:36:08.186913 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1410/2833] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:36:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:36:18.182540 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=32 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:36:19.610854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b0007-3679-462e-84ba-5451d5acb577 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:36:19.611367 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b0007-3679-462e-84ba-5451d5acb577 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:36:19.611367 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b0007-3679-462e-84ba-5451d5acb577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:36:19.630189 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-223b0007-3679-462e-84ba-5451d5acb577 None None] MFA Rules not processed for user `42450ebae9bb445dba460ea02832a173`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:36:19.631379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-223b0007-3679-462e-84ba-5451d5acb577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:36:19.638232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1424/2834] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:36:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:36:19.871674 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07af44-f4ef-4dda-a09b-52a2bf88a4a8 service placement] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:36:19.881444 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07af44-f4ef-4dda-a09b-52a2bf88a4a8 service placement] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:36:19.884987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b70959-a11e-45d6-92fa-914ac437dc96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:36:19.885613 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b70959-a11e-45d6-92fa-914ac437dc96 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:36:19.885904 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b70959-a11e-45d6-92fa-914ac437dc96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:36:19.886165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b70959-a11e-45d6-92fa-914ac437dc96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:36:19.887867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-04b70959-a11e-45d6-92fa-914ac437dc96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:36:19.897907 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b70959-a11e-45d6-92fa-914ac437dc96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:36:19.899313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b70959-a11e-45d6-92fa-914ac437dc96 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:36:19.907777 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1411/2835] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:36:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:36:29.624476 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:36:29.903610 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=6 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:37:01.094277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b0007-3679-462e-84ba-5451d5acb577 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:37:01.095756 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-223b0007-3679-462e-84ba-5451d5acb577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:37:01.107756 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b0007-3679-462e-84ba-5451d5acb577 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:37:01.111562 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb842ef-43ae-4050-983a-2ec3ce83fd1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:37:01.112170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb842ef-43ae-4050-983a-2ec3ce83fd1a service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:37:01.112448 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb842ef-43ae-4050-983a-2ec3ce83fd1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:37:01.112715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb842ef-43ae-4050-983a-2ec3ce83fd1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:37:01.114458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-dfb842ef-43ae-4050-983a-2ec3ce83fd1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:37:01.176005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb842ef-43ae-4050-983a-2ec3ce83fd1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:37:01.177460 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb842ef-43ae-4050-983a-2ec3ce83fd1a service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:37:01.196366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1425/2836] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:37:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:37:05.935417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9c99e-fb26-45c8-9212-baef7f7a71e3 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:37:05.935895 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9c99e-fb26-45c8-9212-baef7f7a71e3 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:37:05.936000 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9c99e-fb26-45c8-9212-baef7f7a71e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:37:05.955544 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-22d9c99e-fb26-45c8-9212-baef7f7a71e3 None None] MFA Rules not processed for user `5b723fcdba1d4b26bd0cf604d59f8d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:37:05.956736 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-22d9c99e-fb26-45c8-9212-baef7f7a71e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:37:06.013229 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1412/2837] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:37:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:37:08.508614 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314a4c7-8101-492c-9b4a-585cb3a71980 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:37:08.509387 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314a4c7-8101-492c-9b4a-585cb3a71980 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:37:08.510085 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314a4c7-8101-492c-9b4a-585cb3a71980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:37:08.530606 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-6314a4c7-8101-492c-9b4a-585cb3a71980 None None] MFA Rules not processed for user `1885a2c4c42b47cb892b097557575642`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:37:08.532414 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-6314a4c7-8101-492c-9b4a-585cb3a71980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:37:08.539162 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1426/2838] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:37:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:37:16.006584 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=25 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:37:18.523749 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=31 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:37:28.653871 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9c99e-fb26-45c8-9212-baef7f7a71e3 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:37:28.655334 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-22d9c99e-fb26-45c8-9212-baef7f7a71e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:37:28.670569 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9c99e-fb26-45c8-9212-baef7f7a71e3 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:37:28.676383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8bbfd-9f6a-4bf8-b0a9-586b9d59de44 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:37:28.677083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8bbfd-9f6a-4bf8-b0a9-586b9d59de44 service placement] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:37:28.677456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8bbfd-9f6a-4bf8-b0a9-586b9d59de44 service placement] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:37:28.677796 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8bbfd-9f6a-4bf8-b0a9-586b9d59de44 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:37:28.679598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1fe8bbfd-9f6a-4bf8-b0a9-586b9d59de44 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:37:28.696582 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8bbfd-9f6a-4bf8-b0a9-586b9d59de44 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:37:28.698987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8bbfd-9f6a-4bf8-b0a9-586b9d59de44 service placement] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:37:28.708830 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314a4c7-8101-492c-9b4a-585cb3a71980 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:37:28.712532 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1413/2839] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:37:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:37:28.728967 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314a4c7-8101-492c-9b4a-585cb3a71980 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:37:28.736032 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f06f8-1b9f-4e22-a133-f309afeb362b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:37:28.736743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f06f8-1b9f-4e22-a133-f309afeb362b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:37:28.737091 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f06f8-1b9f-4e22-a133-f309afeb362b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:37:28.737505 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f06f8-1b9f-4e22-a133-f309afeb362b admin admin] PATH_INFO: `/v3/users/42450ebae9bb445dba460ea02832a173` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:37:28.738851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f06f8-1b9f-4e22-a133-f309afeb362b admin admin] RBAC: Authorizing `identity:delete_user(user_id=42450ebae9bb445dba460ea02832a173)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:37:28.740910 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f06f8-1b9f-4e22-a133-f309afeb362b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:37:28.793945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-fc1f06f8-1b9f-4e22-a133-f309afeb362b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42450ebae9bb445dba460ea02832a173'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:37:28.794689 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-fc1f06f8-1b9f-4e22-a133-f309afeb362b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42450ebae9bb445dba460ea02832a173'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:37:28.802906 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-fc1f06f8-1b9f-4e22-a133-f309afeb362b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42450ebae9bb445dba460ea02832a173'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:37:28.803590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-fc1f06f8-1b9f-4e22-a133-f309afeb362b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42450ebae9bb445dba460ea02832a173'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:37:28.810842 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-fc1f06f8-1b9f-4e22-a133-f309afeb362b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42450ebae9bb445dba460ea02832a173'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:37:28.829064 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1427/2840] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:37:28 2022] DELETE /identity/v3/users/42450ebae9bb445dba460ea02832a173 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:37:29.005822 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8bbfd-9f6a-4bf8-b0a9-586b9d59de44 service placement] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:37:29.021459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8bbfd-9f6a-4bf8-b0a9-586b9d59de44 service placement] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:37:29.093813 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f2341-43df-4aa1-9427-43c3318f44fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:37:29.094858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f2341-43df-4aa1-9427-43c3318f44fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:37:29.095182 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f2341-43df-4aa1-9427-43c3318f44fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:37:29.095615 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f2341-43df-4aa1-9427-43c3318f44fd admin admin] PATH_INFO: `/v3/projects/e6eccd42ceef4259b48149c9835fc73b` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:37:29.097150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f2341-43df-4aa1-9427-43c3318f44fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6eccd42ceef4259b48149c9835fc73b)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:37:29.099669 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f2341-43df-4aa1-9427-43c3318f44fd admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:37:29.147540 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d27f2341-43df-4aa1-9427-43c3318f44fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6eccd42ceef4259b48149c9835fc73b', 'request_id': 'req-d27f2341-43df-4aa1-9427-43c3318f44fd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:37:29.153574 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d27f2341-43df-4aa1-9427-43c3318f44fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6eccd42ceef4259b48149c9835fc73b', 'request_id': 'req-d27f2341-43df-4aa1-9427-43c3318f44fd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:37:29.161448 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d27f2341-43df-4aa1-9427-43c3318f44fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6eccd42ceef4259b48149c9835fc73b', 'request_id': 'req-d27f2341-43df-4aa1-9427-43c3318f44fd'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:37:29.174801 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d27f2341-43df-4aa1-9427-43c3318f44fd admin admin] The token cache is being invalidate because project e6eccd42ceef4259b48149c9835fc73b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:37:29.175145 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d27f2341-43df-4aa1-9427-43c3318f44fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:37:29.175739 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d27f2341-43df-4aa1-9427-43c3318f44fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:37:29.177388 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1414/2841] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:37:29 2022] DELETE /identity/v3/projects/e6eccd42ceef4259b48149c9835fc73b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:37:38.825370 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:37:39.171253 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:38:43.471758 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f06f8-1b9f-4e22-a133-f309afeb362b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:38:43.473923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-fc1f06f8-1b9f-4e22-a133-f309afeb362b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:38:43.530433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f06f8-1b9f-4e22-a133-f309afeb362b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:38:43.577563 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0fc8b-4118-4c3c-995b-86d7b35ed137 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:38:43.578160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0fc8b-4118-4c3c-995b-86d7b35ed137 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:38:43.578436 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0fc8b-4118-4c3c-995b-86d7b35ed137 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:38:43.578707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0fc8b-4118-4c3c-995b-86d7b35ed137 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:38:43.580429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-28c0fc8b-4118-4c3c-995b-86d7b35ed137 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:38:43.648703 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0fc8b-4118-4c3c-995b-86d7b35ed137 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:38:43.650136 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0fc8b-4118-4c3c-995b-86d7b35ed137 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:38:43.661042 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1428/2842] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:38:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:38:52.889576 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f2341-43df-4aa1-9427-43c3318f44fd admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:38:52.890965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d27f2341-43df-4aa1-9427-43c3318f44fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:38:52.903041 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f2341-43df-4aa1-9427-43c3318f44fd admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:38:52.906640 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f2928-0e9a-43ef-8976-252d266c0ce4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:38:52.907269 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f2928-0e9a-43ef-8976-252d266c0ce4 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:38:52.907549 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f2928-0e9a-43ef-8976-252d266c0ce4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:38:52.907820 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f2928-0e9a-43ef-8976-252d266c0ce4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:38:52.909523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d48f2928-0e9a-43ef-8976-252d266c0ce4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:38:52.922970 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f2928-0e9a-43ef-8976-252d266c0ce4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:38:52.924374 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f2928-0e9a-43ef-8976-252d266c0ce4 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:38:52.936018 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1415/2843] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:38:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:38:53.655992 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=74 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:38:57.819854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0fc8b-4118-4c3c-995b-86d7b35ed137 service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:38:57.821399 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-28c0fc8b-4118-4c3c-995b-86d7b35ed137 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:38:57.876561 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0fc8b-4118-4c3c-995b-86d7b35ed137 service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:38:57.884829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b125f7e-f910-4b56-a920-4f7502530918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:38:57.885469 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b125f7e-f910-4b56-a920-4f7502530918 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:38:57.885747 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b125f7e-f910-4b56-a920-4f7502530918 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:38:57.886015 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b125f7e-f910-4b56-a920-4f7502530918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:38:57.887731 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2b125f7e-f910-4b56-a920-4f7502530918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:38:57.897959 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b125f7e-f910-4b56-a920-4f7502530918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:38:57.899346 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b125f7e-f910-4b56-a920-4f7502530918 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:38:57.907153 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1429/2844] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:38:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:39:02.931174 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=6 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:39:07.903613 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=26 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:39:12.907828 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f2928-0e9a-43ef-8976-252d266c0ce4 service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:12.909268 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-d48f2928-0e9a-43ef-8976-252d266c0ce4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:12.921351 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f2928-0e9a-43ef-8976-252d266c0ce4 service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:12.924924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae6312-cdc2-430e-b9d7-504e350b4b4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:12.925551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6312-cdc2-430e-b9d7-504e350b4b4e service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:12.925830 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6312-cdc2-430e-b9d7-504e350b4b4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:12.926103 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6312-cdc2-430e-b9d7-504e350b4b4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:12.928536 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-08ae6312-cdc2-430e-b9d7-504e350b4b4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:12.991517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae6312-cdc2-430e-b9d7-504e350b4b4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:12.992965 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae6312-cdc2-430e-b9d7-504e350b4b4e service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:13.002419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1416/2845] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:39:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:39:16.039587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b125f7e-f910-4b56-a920-4f7502530918 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:16.041006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2b125f7e-f910-4b56-a920-4f7502530918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:16.093061 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae6312-cdc2-430e-b9d7-504e350b4b4e service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:16.095384 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-08ae6312-cdc2-430e-b9d7-504e350b4b4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:16.110094 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae6312-cdc2-430e-b9d7-504e350b4b4e service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:16.115645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f7b26-c3f4-4158-b0bc-03d6268d5778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:16.116356 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f7b26-c3f4-4158-b0bc-03d6268d5778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:16.116631 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f7b26-c3f4-4158-b0bc-03d6268d5778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:16.116903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f7b26-c3f4-4158-b0bc-03d6268d5778 admin admin] PATH_INFO: `/v3/users/1885a2c4c42b47cb892b097557575642` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:16.118213 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f7b26-c3f4-4158-b0bc-03d6268d5778 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1885a2c4c42b47cb892b097557575642)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:16.120142 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f7b26-c3f4-4158-b0bc-03d6268d5778 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:16.125742 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b125f7e-f910-4b56-a920-4f7502530918 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:16.131523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166e7eb-7fc2-48bc-9f41-4a72d54a673d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:16.132272 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166e7eb-7fc2-48bc-9f41-4a72d54a673d service placement] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:16.132554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166e7eb-7fc2-48bc-9f41-4a72d54a673d service placement] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:16.132827 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4166e7eb-7fc2-48bc-9f41-4a72d54a673d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:16.134856 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-4166e7eb-7fc2-48bc-9f41-4a72d54a673d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:16.148379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166e7eb-7fc2-48bc-9f41-4a72d54a673d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:16.150020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4166e7eb-7fc2-48bc-9f41-4a72d54a673d service placement] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:16.160680 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1430/2846] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:39:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:39:16.168497 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d46f7b26-c3f4-4158-b0bc-03d6268d5778 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1885a2c4c42b47cb892b097557575642'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:16.169412 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d46f7b26-c3f4-4158-b0bc-03d6268d5778 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1885a2c4c42b47cb892b097557575642'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:16.187174 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d46f7b26-c3f4-4158-b0bc-03d6268d5778 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1885a2c4c42b47cb892b097557575642'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:16.194890 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d46f7b26-c3f4-4158-b0bc-03d6268d5778 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1885a2c4c42b47cb892b097557575642'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:16.195780 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-d46f7b26-c3f4-4158-b0bc-03d6268d5778 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1885a2c4c42b47cb892b097557575642'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:16.208022 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1417/2847] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:39:16 2022] DELETE /identity/v3/users/1885a2c4c42b47cb892b097557575642 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:39:16.381195 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166e7eb-7fc2-48bc-9f41-4a72d54a673d service placement] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:16.392264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4166e7eb-7fc2-48bc-9f41-4a72d54a673d service placement] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:16.449196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792eb15-f0c6-463c-b030-91d75569ec99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:16.449770 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792eb15-f0c6-463c-b030-91d75569ec99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:16.450053 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792eb15-f0c6-463c-b030-91d75569ec99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:16.450326 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792eb15-f0c6-463c-b030-91d75569ec99 admin admin] PATH_INFO: `/v3/projects/77a0da6cfbd9415cbff856f9cfffee97` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:16.451431 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792eb15-f0c6-463c-b030-91d75569ec99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77a0da6cfbd9415cbff856f9cfffee97)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:16.453376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792eb15-f0c6-463c-b030-91d75569ec99 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:16.498893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d792eb15-f0c6-463c-b030-91d75569ec99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77a0da6cfbd9415cbff856f9cfffee97', 'request_id': 'req-d792eb15-f0c6-463c-b030-91d75569ec99'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:16.504828 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d792eb15-f0c6-463c-b030-91d75569ec99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77a0da6cfbd9415cbff856f9cfffee97', 'request_id': 'req-d792eb15-f0c6-463c-b030-91d75569ec99'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:16.512757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d792eb15-f0c6-463c-b030-91d75569ec99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77a0da6cfbd9415cbff856f9cfffee97', 'request_id': 'req-d792eb15-f0c6-463c-b030-91d75569ec99'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:16.523107 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d792eb15-f0c6-463c-b030-91d75569ec99 admin admin] The token cache is being invalidate because project 77a0da6cfbd9415cbff856f9cfffee97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:39:16.523392 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d792eb15-f0c6-463c-b030-91d75569ec99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:16.524011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-d792eb15-f0c6-463c-b030-91d75569ec99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:16.525534 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1431/2848] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:39:16 2022] DELETE /identity/v3/projects/77a0da6cfbd9415cbff856f9cfffee97 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:39:16.530459 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aced72-cc4b-472a-8eb8-9df66ec5ee39 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:16.530741 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aced72-cc4b-472a-8eb8-9df66ec5ee39 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:16.531013 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aced72-cc4b-472a-8eb8-9df66ec5ee39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:16.550629 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-40aced72-cc4b-472a-8eb8-9df66ec5ee39 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:39:16.551808 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-40aced72-cc4b-472a-8eb8-9df66ec5ee39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:16.615978 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1418/2849] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:39:16.624211 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d239d-a279-4cbc-8e18-4da3baf2a57d None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:16.624564 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d239d-a279-4cbc-8e18-4da3baf2a57d None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:16.624903 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d239d-a279-4cbc-8e18-4da3baf2a57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:16.645147 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-4b5d239d-a279-4cbc-8e18-4da3baf2a57d None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:39:16.646393 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-4b5d239d-a279-4cbc-8e18-4da3baf2a57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:16.654299 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1432/2850] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:39:16.658516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aced72-cc4b-472a-8eb8-9df66ec5ee39 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:16.668818 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aced72-cc4b-472a-8eb8-9df66ec5ee39 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:16.673473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04d3bc-d45e-495c-a6b0-daa66720631d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:16.674074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04d3bc-d45e-495c-a6b0-daa66720631d admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:16.674354 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04d3bc-d45e-495c-a6b0-daa66720631d admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:16.674623 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04d3bc-d45e-495c-a6b0-daa66720631d admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:16.675267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04d3bc-d45e-495c-a6b0-daa66720631d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:39:16.675572 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04d3bc-d45e-495c-a6b0-daa66720631d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:16.677181 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04d3bc-d45e-495c-a6b0-daa66720631d admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:16.679598 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:39:16.692924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1419/2851] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 12:39:16.696665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d239d-a279-4cbc-8e18-4da3baf2a57d None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:16.705891 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d239d-a279-4cbc-8e18-4da3baf2a57d None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:16.710696 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f55b02-38d7-470d-9e78-27de4c57720c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:16.711324 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f55b02-38d7-470d-9e78-27de4c57720c admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:16.711611 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f55b02-38d7-470d-9e78-27de4c57720c admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:16.711873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f55b02-38d7-470d-9e78-27de4c57720c admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:16.712498 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f55b02-38d7-470d-9e78-27de4c57720c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:16.714407 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f55b02-38d7-470d-9e78-27de4c57720c admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:16.726559 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:39:16.744283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1433/2852] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:39:16 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:39:16.748440 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04d3bc-d45e-495c-a6b0-daa66720631d admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:16.758419 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04d3bc-d45e-495c-a6b0-daa66720631d admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:16.813574 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d390afd-ba09-4e0c-95df-70c2b2665bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:16.814176 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d390afd-ba09-4e0c-95df-70c2b2665bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:16.814457 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d390afd-ba09-4e0c-95df-70c2b2665bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:16.814729 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d390afd-ba09-4e0c-95df-70c2b2665bb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:16.815352 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d390afd-ba09-4e0c-95df-70c2b2665bb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:16.817267 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d390afd-ba09-4e0c-95df-70c2b2665bb5 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:16.838375 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1420/2853] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:39:16 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:39:16.842783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f55b02-38d7-470d-9e78-27de4c57720c admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:16.853440 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f55b02-38d7-470d-9e78-27de4c57720c admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:16.857994 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d6690-aca1-4b57-b9b8-1a9539dfc2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:16.858588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d6690-aca1-4b57-b9b8-1a9539dfc2b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:16.858868 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d6690-aca1-4b57-b9b8-1a9539dfc2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:16.859140 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d6690-aca1-4b57-b9b8-1a9539dfc2b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:16.859769 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d6690-aca1-4b57-b9b8-1a9539dfc2b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:16.861367 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d6690-aca1-4b57-b9b8-1a9539dfc2b7 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:16.871838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1434/2854] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:39:16.875519 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d390afd-ba09-4e0c-95df-70c2b2665bb5 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:16.885827 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d390afd-ba09-4e0c-95df-70c2b2665bb5 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:16.890885 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1dd1f-790e-4276-8815-6d3b9ee6d4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:16.891509 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1dd1f-790e-4276-8815-6d3b9ee6d4f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:16.891775 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1dd1f-790e-4276-8815-6d3b9ee6d4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:16.892050 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1dd1f-790e-4276-8815-6d3b9ee6d4f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:16.892742 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea1dd1f-790e-4276-8815-6d3b9ee6d4f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:16.894361 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea1dd1f-790e-4276-8815-6d3b9ee6d4f9 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:16.905347 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1421/2855] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:39:16.909540 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d6690-aca1-4b57-b9b8-1a9539dfc2b7 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:16.920083 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d6690-aca1-4b57-b9b8-1a9539dfc2b7 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:16.924660 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dda955-ed87-43fb-9eb0-4ac94c0c98fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:16.925337 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dda955-ed87-43fb-9eb0-4ac94c0c98fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:16.925622 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dda955-ed87-43fb-9eb0-4ac94c0c98fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:16.925896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dda955-ed87-43fb-9eb0-4ac94c0c98fa admin admin] PATH_INFO: `/v3/projects/0c5c56dbe064490e9b5cac5c2dffe7ac/users/ec194588607849b082ee07caa03c50a5/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:16.940313 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dda955-ed87-43fb-9eb0-4ac94c0c98fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c5c56dbe064490e9b5cac5c2dffe7ac, user_id=ec194588607849b082ee07caa03c50a5, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:16.943024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dda955-ed87-43fb-9eb0-4ac94c0c98fa admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:16.955443 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1435/2856] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:39:16 2022] PUT /identity/v3/projects/0c5c56dbe064490e9b5cac5c2dffe7ac/users/ec194588607849b082ee07caa03c50a5/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 12:39:16.966522 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6a8bf-339a-48d2-ab5c-f8fff95fdd33 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:16.966810 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6a8bf-339a-48d2-ab5c-f8fff95fdd33 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:16.967083 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6a8bf-339a-48d2-ab5c-f8fff95fdd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:16.995071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-1dd6a8bf-339a-48d2-ab5c-f8fff95fdd33 None None] MFA Rules not processed for user `ec194588607849b082ee07caa03c50a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:39:16.996270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1dd6a8bf-339a-48d2-ab5c-f8fff95fdd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:17.058498 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1422/2857] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:39:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:39:17.070245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dda955-ed87-43fb-9eb0-4ac94c0c98fa admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:17.071909 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-96dda955-ed87-43fb-9eb0-4ac94c0c98fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:17.143926 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dda955-ed87-43fb-9eb0-4ac94c0c98fa admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:17.149641 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999bc25-f9f2-4a82-a6b0-793bcc0bb74b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:17.150247 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999bc25-f9f2-4a82-a6b0-793bcc0bb74b service nova] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:17.150527 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999bc25-f9f2-4a82-a6b0-793bcc0bb74b service nova] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:17.150798 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999bc25-f9f2-4a82-a6b0-793bcc0bb74b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:17.158446 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2999bc25-f9f2-4a82-a6b0-793bcc0bb74b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:17.160006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2999bc25-f9f2-4a82-a6b0-793bcc0bb74b service nova] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:17.169152 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1436/2858] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:39:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:39:25.496731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6a8bf-339a-48d2-ab5c-f8fff95fdd33 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:25.558016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6a8bf-339a-48d2-ab5c-f8fff95fdd33 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:25.562638 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624787e6-52f9-457e-8733-bc3fc0b8d664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:25.563360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624787e6-52f9-457e-8733-bc3fc0b8d664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:25.563676 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624787e6-52f9-457e-8733-bc3fc0b8d664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:25.564090 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624787e6-52f9-457e-8733-bc3fc0b8d664 admin admin] PATH_INFO: `/v3/users/ec194588607849b082ee07caa03c50a5` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:25.565122 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624787e6-52f9-457e-8733-bc3fc0b8d664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec194588607849b082ee07caa03c50a5)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:25.567076 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624787e6-52f9-457e-8733-bc3fc0b8d664 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:25.570066 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:39:25.614477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-624787e6-52f9-457e-8733-bc3fc0b8d664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec194588607849b082ee07caa03c50a5'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:25.615133 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-624787e6-52f9-457e-8733-bc3fc0b8d664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec194588607849b082ee07caa03c50a5'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:25.627032 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-624787e6-52f9-457e-8733-bc3fc0b8d664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec194588607849b082ee07caa03c50a5'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:25.634699 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-624787e6-52f9-457e-8733-bc3fc0b8d664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec194588607849b082ee07caa03c50a5'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:25.635303 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-624787e6-52f9-457e-8733-bc3fc0b8d664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec194588607849b082ee07caa03c50a5'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:25.647883 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1423/2859] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:39:25 2022] DELETE /identity/v3/users/ec194588607849b082ee07caa03c50a5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:39:25.652364 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dab69-dc08-4610-8544-faeff84aeeed None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:25.652858 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dab69-dc08-4610-8544-faeff84aeeed None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:25.652963 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dab69-dc08-4610-8544-faeff84aeeed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:25.671296 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-b77dab69-dc08-4610-8544-faeff84aeeed None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:39:25.672490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b77dab69-dc08-4610-8544-faeff84aeeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:25.732530 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1437/2860] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:39:25.743105 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624787e6-52f9-457e-8733-bc3fc0b8d664 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:25.744507 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-624787e6-52f9-457e-8733-bc3fc0b8d664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:25.800467 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624787e6-52f9-457e-8733-bc3fc0b8d664 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:25.804897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602b447-592d-4388-8349-d8230c33c717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:25.805546 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602b447-592d-4388-8349-d8230c33c717 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:25.805829 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602b447-592d-4388-8349-d8230c33c717 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:25.806101 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602b447-592d-4388-8349-d8230c33c717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:25.815069 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602b447-592d-4388-8349-d8230c33c717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:25.816516 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602b447-592d-4388-8349-d8230c33c717 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:25.826150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1424/2861] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:39:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:39:26.093006 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dab69-dc08-4610-8544-faeff84aeeed None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:26.094598 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-b77dab69-dc08-4610-8544-faeff84aeeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:26.106342 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dab69-dc08-4610-8544-faeff84aeeed None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:26.110852 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc71d5-64fd-4b17-88bb-a1ca2915a2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:26.111492 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc71d5-64fd-4b17-88bb-a1ca2915a2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:26.111770 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc71d5-64fd-4b17-88bb-a1ca2915a2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:26.112037 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc71d5-64fd-4b17-88bb-a1ca2915a2da admin admin] PATH_INFO: `/v3/projects/0c5c56dbe064490e9b5cac5c2dffe7ac` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:26.113154 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc71d5-64fd-4b17-88bb-a1ca2915a2da admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c5c56dbe064490e9b5cac5c2dffe7ac)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:26.115711 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc71d5-64fd-4b17-88bb-a1ca2915a2da admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:26.153714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:39:26.164500 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5ddc71d5-64fd-4b17-88bb-a1ca2915a2da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c5c56dbe064490e9b5cac5c2dffe7ac', 'request_id': 'req-5ddc71d5-64fd-4b17-88bb-a1ca2915a2da'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:26.170666 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5ddc71d5-64fd-4b17-88bb-a1ca2915a2da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c5c56dbe064490e9b5cac5c2dffe7ac', 'request_id': 'req-5ddc71d5-64fd-4b17-88bb-a1ca2915a2da'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:26.179571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5ddc71d5-64fd-4b17-88bb-a1ca2915a2da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c5c56dbe064490e9b5cac5c2dffe7ac', 'request_id': 'req-5ddc71d5-64fd-4b17-88bb-a1ca2915a2da'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:26.190433 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5ddc71d5-64fd-4b17-88bb-a1ca2915a2da admin admin] The token cache is being invalidate because project 0c5c56dbe064490e9b5cac5c2dffe7ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:39:26.190719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5ddc71d5-64fd-4b17-88bb-a1ca2915a2da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:26.191351 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-5ddc71d5-64fd-4b17-88bb-a1ca2915a2da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:26.193667 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1438/2862] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:39:26 2022] DELETE /identity/v3/projects/0c5c56dbe064490e9b5cac5c2dffe7ac => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:39:26.199009 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23c0f4-253c-4900-ae5c-e93c32bbd017 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:26.199293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23c0f4-253c-4900-ae5c-e93c32bbd017 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:26.199566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23c0f4-253c-4900-ae5c-e93c32bbd017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:26.220220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-fc23c0f4-253c-4900-ae5c-e93c32bbd017 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:39:26.221379 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-fc23c0f4-253c-4900-ae5c-e93c32bbd017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:26.283132 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1425/2863] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:39:26.292829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03b603-b136-4493-a784-2f2753dd3125 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:26.293155 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03b603-b136-4493-a784-2f2753dd3125 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:26.293414 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e03b603-b136-4493-a784-2f2753dd3125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:26.312783 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-5e03b603-b136-4493-a784-2f2753dd3125 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:39:26.313997 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5e03b603-b136-4493-a784-2f2753dd3125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:26.322359 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1439/2864] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:39:26.326721 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23c0f4-253c-4900-ae5c-e93c32bbd017 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:26.336823 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23c0f4-253c-4900-ae5c-e93c32bbd017 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:26.341635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d4bc8-b4cd-4ec7-8873-a991ce9f6395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:26.342226 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d4bc8-b4cd-4ec7-8873-a991ce9f6395 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:26.342501 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d4bc8-b4cd-4ec7-8873-a991ce9f6395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:26.342766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d4bc8-b4cd-4ec7-8873-a991ce9f6395 admin admin] PATH_INFO: `/v3/domains` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:26.343400 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d4bc8-b4cd-4ec7-8873-a991ce9f6395 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81385) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:39:26.343701 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d4bc8-b4cd-4ec7-8873-a991ce9f6395 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:26.345306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d4bc8-b4cd-4ec7-8873-a991ce9f6395 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:26.356548 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1426/2865] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:39:26.360739 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03b603-b136-4493-a784-2f2753dd3125 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:26.369893 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e03b603-b136-4493-a784-2f2753dd3125 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:26.374501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271dc4a-0913-450c-a31a-9f352b56de48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:26.375090 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271dc4a-0913-450c-a31a-9f352b56de48 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:26.375364 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271dc4a-0913-450c-a31a-9f352b56de48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:26.375628 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271dc4a-0913-450c-a31a-9f352b56de48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:26.376250 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271dc4a-0913-450c-a31a-9f352b56de48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:26.378172 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271dc4a-0913-450c-a31a-9f352b56de48 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:26.405196 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1440/2866] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:39:26 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:39:26.409357 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d4bc8-b4cd-4ec7-8873-a991ce9f6395 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:26.419452 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d4bc8-b4cd-4ec7-8873-a991ce9f6395 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:26.427157 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:39:26.477491 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8dcc30-594a-4a52-86f3-1f40c0c49b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:26.478098 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dcc30-594a-4a52-86f3-1f40c0c49b30 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:26.478382 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dcc30-594a-4a52-86f3-1f40c0c49b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:26.478657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8dcc30-594a-4a52-86f3-1f40c0c49b30 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:26.479924 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8dcc30-594a-4a52-86f3-1f40c0c49b30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:26.481970 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8dcc30-594a-4a52-86f3-1f40c0c49b30 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:26.502201 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1427/2867] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:39:26 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:39:26.506862 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271dc4a-0913-450c-a31a-9f352b56de48 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:26.517072 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271dc4a-0913-450c-a31a-9f352b56de48 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:26.521811 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99608e-e222-4af0-ae56-7cce3dcc265f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:26.522405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99608e-e222-4af0-ae56-7cce3dcc265f admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:26.522683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99608e-e222-4af0-ae56-7cce3dcc265f admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:26.522945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99608e-e222-4af0-ae56-7cce3dcc265f admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:26.523573 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc99608e-e222-4af0-ae56-7cce3dcc265f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:26.525201 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc99608e-e222-4af0-ae56-7cce3dcc265f admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:26.535047 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1441/2868] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 12:39:26.539578 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8dcc30-594a-4a52-86f3-1f40c0c49b30 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:26.549337 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8dcc30-594a-4a52-86f3-1f40c0c49b30 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:26.554597 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead71539-5e06-4924-840e-faf2c40ea628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:26.555185 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead71539-5e06-4924-840e-faf2c40ea628 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:26.555465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead71539-5e06-4924-840e-faf2c40ea628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:26.555734 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead71539-5e06-4924-840e-faf2c40ea628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:26.556359 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead71539-5e06-4924-840e-faf2c40ea628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:26.557976 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead71539-5e06-4924-840e-faf2c40ea628 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:26.569055 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1428/2869] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:39:26.572758 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99608e-e222-4af0-ae56-7cce3dcc265f admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:26.582566 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99608e-e222-4af0-ae56-7cce3dcc265f admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:26.587829 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10673f24-dd01-4357-8f4b-126f293d654b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:26.588463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10673f24-dd01-4357-8f4b-126f293d654b admin admin] REQUEST_METHOD: `PUT` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:26.588751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10673f24-dd01-4357-8f4b-126f293d654b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:26.589062 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10673f24-dd01-4357-8f4b-126f293d654b admin admin] PATH_INFO: `/v3/projects/3824a31f4388419c82f9981614932b65/users/83b1b0a0ed634c6b99b1f3e365c1d415/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:26.603393 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10673f24-dd01-4357-8f4b-126f293d654b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3824a31f4388419c82f9981614932b65, user_id=83b1b0a0ed634c6b99b1f3e365c1d415, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:26.606072 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10673f24-dd01-4357-8f4b-126f293d654b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:26.620281 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1442/2870] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:39:26 2022] PUT /identity/v3/projects/3824a31f4388419c82f9981614932b65/users/83b1b0a0ed634c6b99b1f3e365c1d415/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:39:26.681426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd3eda-9ccf-4f22-80c0-bbd0cf4f627e None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:26.681712 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd3eda-9ccf-4f22-80c0-bbd0cf4f627e None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:26.681987 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd3eda-9ccf-4f22-80c0-bbd0cf4f627e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:26.710074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-b6fd3eda-9ccf-4f22-80c0-bbd0cf4f627e None None] MFA Rules not processed for user `83b1b0a0ed634c6b99b1f3e365c1d415`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:39:26.711282 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-b6fd3eda-9ccf-4f22-80c0-bbd0cf4f627e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:26.775826 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1429/2871] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:39:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:39:26.780326 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10673f24-dd01-4357-8f4b-126f293d654b admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:26.842496 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10673f24-dd01-4357-8f4b-126f293d654b admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:26.847076 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e090a-f26c-496a-a4f1-b33de42a763d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:26.847676 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e090a-f26c-496a-a4f1-b33de42a763d admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:26.847958 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e090a-f26c-496a-a4f1-b33de42a763d admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:26.848231 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e090a-f26c-496a-a4f1-b33de42a763d admin admin] PATH_INFO: `/v3/projects` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:26.849024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e090a-f26c-496a-a4f1-b33de42a763d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:26.851098 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e090a-f26c-496a-a4f1-b33de42a763d admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:26.877330 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1443/2872] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:39:26 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:39:26.885897 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd3eda-9ccf-4f22-80c0-bbd0cf4f627e None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:26.896903 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd3eda-9ccf-4f22-80c0-bbd0cf4f627e None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:26.952302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d75f73-a609-4bca-861c-fc9ea674f851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:26.952905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d75f73-a609-4bca-861c-fc9ea674f851 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:26.953386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d75f73-a609-4bca-861c-fc9ea674f851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:26.953677 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d75f73-a609-4bca-861c-fc9ea674f851 admin admin] PATH_INFO: `/v3/users` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:26.954298 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d75f73-a609-4bca-861c-fc9ea674f851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:26.956184 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d75f73-a609-4bca-861c-fc9ea674f851 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:26.977259 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1430/2873] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:39:26 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:39:26.981779 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e090a-f26c-496a-a4f1-b33de42a763d admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:26.992920 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e090a-f26c-496a-a4f1-b33de42a763d admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:26.997567 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3e6e4-aeb5-47d1-a148-75a98d172011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:26.998167 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3e6e4-aeb5-47d1-a148-75a98d172011 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:26.998465 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3e6e4-aeb5-47d1-a148-75a98d172011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:26.998723 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3e6e4-aeb5-47d1-a148-75a98d172011 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:26.999354 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3e6e4-aeb5-47d1-a148-75a98d172011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:27.000928 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3e6e4-aeb5-47d1-a148-75a98d172011 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:27.012173 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1444/2874] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:39:27.016190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d75f73-a609-4bca-861c-fc9ea674f851 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:27.026023 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d75f73-a609-4bca-861c-fc9ea674f851 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:27.030623 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc1122-9f22-4d67-9a9e-b066fc614d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:27.031263 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc1122-9f22-4d67-9a9e-b066fc614d25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:27.031539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc1122-9f22-4d67-9a9e-b066fc614d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:27.031809 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc1122-9f22-4d67-9a9e-b066fc614d25 admin admin] PATH_INFO: `/v3/projects/b376cf7fe10f414a96e39d0b034d6245/users/c54032d5f4d74d41a3ff7a731165190c/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:27.034772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:39:27.047911 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc1122-9f22-4d67-9a9e-b066fc614d25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b376cf7fe10f414a96e39d0b034d6245, user_id=c54032d5f4d74d41a3ff7a731165190c, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:27.050552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc1122-9f22-4d67-9a9e-b066fc614d25 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:27.064003 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1431/2875] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:39:27 2022] PUT /identity/v3/projects/b376cf7fe10f414a96e39d0b034d6245/users/c54032d5f4d74d41a3ff7a731165190c/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:39:27.067957 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3e6e4-aeb5-47d1-a148-75a98d172011 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:27.078065 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3e6e4-aeb5-47d1-a148-75a98d172011 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:27.110232 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:39:27.136624 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29616779-796d-4c23-87b1-ff840f381e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:27.137269 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29616779-796d-4c23-87b1-ff840f381e34 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:27.137551 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29616779-796d-4c23-87b1-ff840f381e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:27.137842 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29616779-796d-4c23-87b1-ff840f381e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:27.138458 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29616779-796d-4c23-87b1-ff840f381e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:27.140379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29616779-796d-4c23-87b1-ff840f381e34 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:27.150524 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1445/2876] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:39:27.154706 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc1122-9f22-4d67-9a9e-b066fc614d25 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:27.165331 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc1122-9f22-4d67-9a9e-b066fc614d25 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:27.171401 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3660e-6886-481b-8705-008f182486b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:27.172377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3660e-6886-481b-8705-008f182486b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:27.172377 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3660e-6886-481b-8705-008f182486b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:27.172604 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3660e-6886-481b-8705-008f182486b7 admin admin] PATH_INFO: `/v3/projects/b376cf7fe10f414a96e39d0b034d6245/users/c54032d5f4d74d41a3ff7a731165190c/roles/c11f371c53dd40aab2cdc96e05805715` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:27.174658 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3660e-6886-481b-8705-008f182486b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b376cf7fe10f414a96e39d0b034d6245, user_id=c54032d5f4d74d41a3ff7a731165190c, role_id=c11f371c53dd40aab2cdc96e05805715)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:27.178048 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3660e-6886-481b-8705-008f182486b7 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:27.188399 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1432/2877] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:39:27 2022] PUT /identity/v3/projects/b376cf7fe10f414a96e39d0b034d6245/users/c54032d5f4d74d41a3ff7a731165190c/roles/c11f371c53dd40aab2cdc96e05805715 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:39:27.198495 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe565836-a867-43fc-8c19-3edf919a8c19 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:27.198787 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe565836-a867-43fc-8c19-3edf919a8c19 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:27.199065 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe565836-a867-43fc-8c19-3edf919a8c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:27.226960 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-fe565836-a867-43fc-8c19-3edf919a8c19 None None] MFA Rules not processed for user `c54032d5f4d74d41a3ff7a731165190c`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:39:27.228158 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-fe565836-a867-43fc-8c19-3edf919a8c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:27.295265 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1446/2878] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:39:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:39:27.307276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3660e-6886-481b-8705-008f182486b7 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:27.308991 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-20b3660e-6886-481b-8705-008f182486b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:27.368289 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3660e-6886-481b-8705-008f182486b7 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:27.371839 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfe26b-0819-4243-8331-ef844d2311f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:27.372437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfe26b-0819-4243-8331-ef844d2311f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:27.372720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfe26b-0819-4243-8331-ef844d2311f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:27.372994 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfe26b-0819-4243-8331-ef844d2311f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:27.382285 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfe26b-0819-4243-8331-ef844d2311f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:27.383698 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfe26b-0819-4243-8331-ef844d2311f4 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:27.393118 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1433/2879] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:39:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:39:27.460897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe565836-a867-43fc-8c19-3edf919a8c19 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:27.522245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe565836-a867-43fc-8c19-3edf919a8c19 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:27.526744 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e385e-cf42-4d82-9d55-8f3768b8de8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:27.527362 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e385e-cf42-4d82-9d55-8f3768b8de8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:27.527638 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e385e-cf42-4d82-9d55-8f3768b8de8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:27.527906 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e385e-cf42-4d82-9d55-8f3768b8de8e admin admin] PATH_INFO: `/v3/users/83b1b0a0ed634c6b99b1f3e365c1d415` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:27.529028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e385e-cf42-4d82-9d55-8f3768b8de8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=83b1b0a0ed634c6b99b1f3e365c1d415)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:27.530926 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e385e-cf42-4d82-9d55-8f3768b8de8e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:27.574677 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-754e385e-cf42-4d82-9d55-8f3768b8de8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83b1b0a0ed634c6b99b1f3e365c1d415'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:27.575314 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-754e385e-cf42-4d82-9d55-8f3768b8de8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83b1b0a0ed634c6b99b1f3e365c1d415'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:27.582082 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-754e385e-cf42-4d82-9d55-8f3768b8de8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83b1b0a0ed634c6b99b1f3e365c1d415'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:27.582701 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-754e385e-cf42-4d82-9d55-8f3768b8de8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83b1b0a0ed634c6b99b1f3e365c1d415'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:27.587724 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:39:27.591537 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-754e385e-cf42-4d82-9d55-8f3768b8de8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83b1b0a0ed634c6b99b1f3e365c1d415'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:27.609160 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1447/2880] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:39:27 2022] DELETE /identity/v3/users/83b1b0a0ed634c6b99b1f3e365c1d415 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:39:27.613636 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa066d-b2bc-41ca-ba7c-e77684b933ff None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:27.614150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa066d-b2bc-41ca-ba7c-e77684b933ff None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:27.614150 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa066d-b2bc-41ca-ba7c-e77684b933ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:27.632450 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-21fa066d-b2bc-41ca-ba7c-e77684b933ff None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:39:27.633602 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-21fa066d-b2bc-41ca-ba7c-e77684b933ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:27.689532 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1434/2881] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:39:27.699283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e385e-cf42-4d82-9d55-8f3768b8de8e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:27.700688 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-754e385e-cf42-4d82-9d55-8f3768b8de8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:27.758263 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e385e-cf42-4d82-9d55-8f3768b8de8e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:27.761851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0e624-cb4a-4ed9-bf1e-f3494c7c1e37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:27.762444 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0e624-cb4a-4ed9-bf1e-f3494c7c1e37 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:27.762719 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0e624-cb4a-4ed9-bf1e-f3494c7c1e37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:27.762983 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0e624-cb4a-4ed9-bf1e-f3494c7c1e37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:27.771765 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0e624-cb4a-4ed9-bf1e-f3494c7c1e37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:27.773860 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0e624-cb4a-4ed9-bf1e-f3494c7c1e37 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:27.783149 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1448/2882] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:39:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:39:28.040765 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa066d-b2bc-41ca-ba7c-e77684b933ff None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:28.042209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-21fa066d-b2bc-41ca-ba7c-e77684b933ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:28.054641 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa066d-b2bc-41ca-ba7c-e77684b933ff None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:28.059221 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:28.059848 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:28.060127 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:28.060425 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c admin admin] PATH_INFO: `/v3/projects/3824a31f4388419c82f9981614932b65` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:28.061551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3824a31f4388419c82f9981614932b65)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:28.063435 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:28.111620 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3824a31f4388419c82f9981614932b65', 'request_id': 'req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:28.118082 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3824a31f4388419c82f9981614932b65', 'request_id': 'req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:28.125024 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3824a31f4388419c82f9981614932b65', 'request_id': 'req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:28.131460 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:39:28.137096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c admin admin] The token cache is being invalidate because project 3824a31f4388419c82f9981614932b65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:39:28.137640 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:28.138634 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:28.140136 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1435/2883] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:39:28 2022] DELETE /identity/v3/projects/3824a31f4388419c82f9981614932b65 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:39:28.144132 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0e624-cb4a-4ed9-bf1e-f3494c7c1e37 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:28.154235 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0e624-cb4a-4ed9-bf1e-f3494c7c1e37 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:28.210548 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de824218-d454-4628-b95f-2e0bdbedf967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:28.211173 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de824218-d454-4628-b95f-2e0bdbedf967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:28.211450 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de824218-d454-4628-b95f-2e0bdbedf967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:28.211722 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de824218-d454-4628-b95f-2e0bdbedf967 admin admin] PATH_INFO: `/v3/users/c54032d5f4d74d41a3ff7a731165190c` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:28.212824 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de824218-d454-4628-b95f-2e0bdbedf967 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c54032d5f4d74d41a3ff7a731165190c)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:28.215013 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de824218-d454-4628-b95f-2e0bdbedf967 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:28.260325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-de824218-d454-4628-b95f-2e0bdbedf967 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c54032d5f4d74d41a3ff7a731165190c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:28.260970 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-de824218-d454-4628-b95f-2e0bdbedf967 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c54032d5f4d74d41a3ff7a731165190c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:28.268024 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-de824218-d454-4628-b95f-2e0bdbedf967 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c54032d5f4d74d41a3ff7a731165190c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:28.268698 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-de824218-d454-4628-b95f-2e0bdbedf967 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c54032d5f4d74d41a3ff7a731165190c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:28.275743 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-de824218-d454-4628-b95f-2e0bdbedf967 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c54032d5f4d74d41a3ff7a731165190c'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:28.288643 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:39:28.294823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1449/2884] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:39:28 2022] DELETE /identity/v3/users/c54032d5f4d74d41a3ff7a731165190c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:39:28.560727 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:28.562212 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:28.575195 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e6bc7-5f7e-41c8-9a0e-4d57aace383c admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:28.633335 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5981e-c273-4b9c-b886-f92a677c5487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:28.634016 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5981e-c273-4b9c-b886-f92a677c5487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:28.634239 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5981e-c273-4b9c-b886-f92a677c5487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:28.634514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5981e-c273-4b9c-b886-f92a677c5487 admin admin] PATH_INFO: `/v3/projects/b376cf7fe10f414a96e39d0b034d6245` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:28.635628 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5981e-c273-4b9c-b886-f92a677c5487 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b376cf7fe10f414a96e39d0b034d6245)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:28.637565 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5981e-c273-4b9c-b886-f92a677c5487 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:28.688881 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-82d5981e-c273-4b9c-b886-f92a677c5487 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b376cf7fe10f414a96e39d0b034d6245', 'request_id': 'req-82d5981e-c273-4b9c-b886-f92a677c5487'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:28.695106 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-82d5981e-c273-4b9c-b886-f92a677c5487 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b376cf7fe10f414a96e39d0b034d6245', 'request_id': 'req-82d5981e-c273-4b9c-b886-f92a677c5487'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:28.702360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-82d5981e-c273-4b9c-b886-f92a677c5487 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b376cf7fe10f414a96e39d0b034d6245', 'request_id': 'req-82d5981e-c273-4b9c-b886-f92a677c5487'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:28.712731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-82d5981e-c273-4b9c-b886-f92a677c5487 admin admin] The token cache is being invalidate because project b376cf7fe10f414a96e39d0b034d6245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:39:28.713027 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-82d5981e-c273-4b9c-b886-f92a677c5487 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:28.713674 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-82d5981e-c273-4b9c-b886-f92a677c5487 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:39:28.715405 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1436/2885] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:39:28 2022] DELETE /identity/v3/projects/b376cf7fe10f414a96e39d0b034d6245 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:39:28.720452 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a5b78-c16b-40c0-a563-9801f59c1b5e None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:28.720734 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a5b78-c16b-40c0-a563-9801f59c1b5e None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:28.721009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a5b78-c16b-40c0-a563-9801f59c1b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:28.742073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-9b9a5b78-c16b-40c0-a563-9801f59c1b5e None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:39:28.743268 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-9b9a5b78-c16b-40c0-a563-9801f59c1b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:28.801260 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1450/2886] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:39:28.810694 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9c1d5-8781-450a-b85e-9b029d1ac70b None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:28.810971 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9c1d5-8781-450a-b85e-9b029d1ac70b None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:28.811234 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9c1d5-8781-450a-b85e-9b029d1ac70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:28.830517 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-bed9c1d5-8781-450a-b85e-9b029d1ac70b None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:39:28.831699 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-bed9c1d5-8781-450a-b85e-9b029d1ac70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:28.840329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1437/2887] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:39:28.844646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a5b78-c16b-40c0-a563-9801f59c1b5e None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:28.855011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a5b78-c16b-40c0-a563-9801f59c1b5e None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:28.859513 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87443b7-aafe-4b4e-b5a2-ea4c050d42ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:28.860110 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87443b7-aafe-4b4e-b5a2-ea4c050d42ed admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:28.860392 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87443b7-aafe-4b4e-b5a2-ea4c050d42ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:28.860668 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87443b7-aafe-4b4e-b5a2-ea4c050d42ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:28.861347 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87443b7-aafe-4b4e-b5a2-ea4c050d42ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:39:28.861655 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87443b7-aafe-4b4e-b5a2-ea4c050d42ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:28.863245 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87443b7-aafe-4b4e-b5a2-ea4c050d42ed admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:28.874618 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1451/2888] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:39:28.878736 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9c1d5-8781-450a-b85e-9b029d1ac70b None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:28.887870 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9c1d5-8781-450a-b85e-9b029d1ac70b None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:28.892591 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb94ab0-6389-486b-8d42-2ab519aceb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:28.893201 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb94ab0-6389-486b-8d42-2ab519aceb78 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:28.893481 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb94ab0-6389-486b-8d42-2ab519aceb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:28.893747 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb94ab0-6389-486b-8d42-2ab519aceb78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:28.894396 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb94ab0-6389-486b-8d42-2ab519aceb78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:28.896248 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb94ab0-6389-486b-8d42-2ab519aceb78 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:28.923365 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1438/2889] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:39:28 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:39:28.927690 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87443b7-aafe-4b4e-b5a2-ea4c050d42ed admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:28.937941 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87443b7-aafe-4b4e-b5a2-ea4c050d42ed admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:28.993910 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c1410-ae28-4c6c-ae80-a56b61cb52a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:28.994451 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c1410-ae28-4c6c-ae80-a56b61cb52a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:28.994735 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c1410-ae28-4c6c-ae80-a56b61cb52a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:28.995011 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c1410-ae28-4c6c-ae80-a56b61cb52a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:28.995647 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c1410-ae28-4c6c-ae80-a56b61cb52a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:28.997590 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c1410-ae28-4c6c-ae80-a56b61cb52a8 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:29.021224 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1452/2890] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:39:28 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:39:29.025994 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb94ab0-6389-486b-8d42-2ab519aceb78 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:29.036223 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb94ab0-6389-486b-8d42-2ab519aceb78 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:29.041581 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f28b4-237a-46b2-9ea9-5f645d65ae4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:29.042179 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f28b4-237a-46b2-9ea9-5f645d65ae4f admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:29.042462 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f28b4-237a-46b2-9ea9-5f645d65ae4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:29.042738 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f28b4-237a-46b2-9ea9-5f645d65ae4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:29.043373 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f28b4-237a-46b2-9ea9-5f645d65ae4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:29.044953 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f28b4-237a-46b2-9ea9-5f645d65ae4f admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:29.054832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1439/2891] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:39:29.058606 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c1410-ae28-4c6c-ae80-a56b61cb52a8 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:29.068325 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c1410-ae28-4c6c-ae80-a56b61cb52a8 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:29.072987 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899bdc4-8804-41f1-8b47-93ee724a5b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:29.073615 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899bdc4-8804-41f1-8b47-93ee724a5b77 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:29.073897 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899bdc4-8804-41f1-8b47-93ee724a5b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:29.074171 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899bdc4-8804-41f1-8b47-93ee724a5b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:29.074805 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899bdc4-8804-41f1-8b47-93ee724a5b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:29.076371 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899bdc4-8804-41f1-8b47-93ee724a5b77 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:29.087272 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1453/2892] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:39:29.091415 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f28b4-237a-46b2-9ea9-5f645d65ae4f admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:29.102518 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f28b4-237a-46b2-9ea9-5f645d65ae4f admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:29.107081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343bb1a-421b-4782-860e-6647cb2ef9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:29.107751 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343bb1a-421b-4782-860e-6647cb2ef9a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:29.108033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343bb1a-421b-4782-860e-6647cb2ef9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:29.108309 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f343bb1a-421b-4782-860e-6647cb2ef9a1 admin admin] PATH_INFO: `/v3/projects/9e7346b5ae604bf39f284fb45d53d617/users/bcb90ae7084041aab512ebdaabcfc782/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:29.122689 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343bb1a-421b-4782-860e-6647cb2ef9a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e7346b5ae604bf39f284fb45d53d617, user_id=bcb90ae7084041aab512ebdaabcfc782, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:29.125329 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f343bb1a-421b-4782-860e-6647cb2ef9a1 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:29.139543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1440/2893] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:39:29 2022] PUT /identity/v3/projects/9e7346b5ae604bf39f284fb45d53d617/users/bcb90ae7084041aab512ebdaabcfc782/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:39:29.150612 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848c5ec-4aff-400c-83b3-d5dfe7f085a4 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:29.150899 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848c5ec-4aff-400c-83b3-d5dfe7f085a4 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:29.151173 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848c5ec-4aff-400c-83b3-d5dfe7f085a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:29.179370 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-e848c5ec-4aff-400c-83b3-d5dfe7f085a4 None None] MFA Rules not processed for user `bcb90ae7084041aab512ebdaabcfc782`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:39:29.180589 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-e848c5ec-4aff-400c-83b3-d5dfe7f085a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:29.234406 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:39:29.246823 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1454/2894] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:39:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:39:29.258271 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343bb1a-421b-4782-860e-6647cb2ef9a1 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:29.259972 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-f343bb1a-421b-4782-860e-6647cb2ef9a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:29.287938 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:39:29.320603 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f343bb1a-421b-4782-860e-6647cb2ef9a1 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:29.324190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c08426-06c3-436e-9d07-eedeaa11cfec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:29.324792 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c08426-06c3-436e-9d07-eedeaa11cfec service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:29.325070 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c08426-06c3-436e-9d07-eedeaa11cfec service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:29.325371 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c08426-06c3-436e-9d07-eedeaa11cfec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:29.333480 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c08426-06c3-436e-9d07-eedeaa11cfec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:29.335503 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c08426-06c3-436e-9d07-eedeaa11cfec service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:29.344363 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1441/2895] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:39:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:39:32.218315 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848c5ec-4aff-400c-83b3-d5dfe7f085a4 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:32.228646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848c5ec-4aff-400c-83b3-d5dfe7f085a4 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:32.232206 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b32384-9157-4b15-b7aa-1cf4164fa659 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:32.232806 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b32384-9157-4b15-b7aa-1cf4164fa659 service cinder] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:32.233088 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b32384-9157-4b15-b7aa-1cf4164fa659 service cinder] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:32.233397 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b32384-9157-4b15-b7aa-1cf4164fa659 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:32.235101 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-81b32384-9157-4b15-b7aa-1cf4164fa659 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:32.293531 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b32384-9157-4b15-b7aa-1cf4164fa659 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:32.294929 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b32384-9157-4b15-b7aa-1cf4164fa659 service cinder] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:32.304380 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1455/2896] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:39:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:39:39.340476 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=15 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:39:42.299600 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=31 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:39:49.606450 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b161b-1255-47d9-9b29-4f87f4b75ca3 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:49.607034 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b161b-1255-47d9-9b29-4f87f4b75ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:49.607468 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b161b-1255-47d9-9b29-4f87f4b75ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:49.627928 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-e39b161b-1255-47d9-9b29-4f87f4b75ca3 None None] MFA Rules not processed for user `5b723fcdba1d4b26bd0cf604d59f8d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:39:49.629160 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e39b161b-1255-47d9-9b29-4f87f4b75ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:49.636716 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1442/2897] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:39:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:39:50.424312 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b32384-9157-4b15-b7aa-1cf4164fa659 service cinder] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:50.425780 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-81b32384-9157-4b15-b7aa-1cf4164fa659 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:50.500182 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b32384-9157-4b15-b7aa-1cf4164fa659 service cinder] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:50.505388 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb5535-5876-4959-b06b-57fb31e5366b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:50.506477 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb5535-5876-4959-b06b-57fb31e5366b service placement] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:50.506755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb5535-5876-4959-b06b-57fb31e5366b service placement] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:50.507028 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb5535-5876-4959-b06b-57fb31e5366b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:50.508802 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-aefb5535-5876-4959-b06b-57fb31e5366b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:50.520109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb5535-5876-4959-b06b-57fb31e5366b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:50.522262 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb5535-5876-4959-b06b-57fb31e5366b service placement] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:50.532041 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1456/2898] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:39:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:39:56.772321 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b161b-1255-47d9-9b29-4f87f4b75ca3 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:39:56.773879 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-e39b161b-1255-47d9-9b29-4f87f4b75ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:56.862165 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b161b-1255-47d9-9b29-4f87f4b75ca3 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:39:56.867772 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bed878-015e-48b3-a892-731b3bbfb47f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:39:56.868386 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bed878-015e-48b3-a892-731b3bbfb47f service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:39:56.868668 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bed878-015e-48b3-a892-731b3bbfb47f service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:39:56.868939 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bed878-015e-48b3-a892-731b3bbfb47f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:39:56.870844 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-40bed878-015e-48b3-a892-731b3bbfb47f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:39:56.883740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bed878-015e-48b3-a892-731b3bbfb47f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:39:56.885465 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bed878-015e-48b3-a892-731b3bbfb47f service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:39:56.905427 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1443/2899] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:39:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:40:00.527697 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=32 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:40:01.458755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb5535-5876-4959-b06b-57fb31e5366b service placement] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:40:01.523293 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb5535-5876-4959-b06b-57fb31e5366b service placement] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:40:01.527770 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922daf9-d400-477f-89ae-8d1567492c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:40:01.528390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922daf9-d400-477f-89ae-8d1567492c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:01.528665 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922daf9-d400-477f-89ae-8d1567492c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:01.528933 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922daf9-d400-477f-89ae-8d1567492c4e admin admin] PATH_INFO: `/v3/users/bcb90ae7084041aab512ebdaabcfc782` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:01.530073 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922daf9-d400-477f-89ae-8d1567492c4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb90ae7084041aab512ebdaabcfc782)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:40:01.531950 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922daf9-d400-477f-89ae-8d1567492c4e admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:40:01.576333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0922daf9-d400-477f-89ae-8d1567492c4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb90ae7084041aab512ebdaabcfc782'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:40:01.576978 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0922daf9-d400-477f-89ae-8d1567492c4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb90ae7084041aab512ebdaabcfc782'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:40:01.584016 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0922daf9-d400-477f-89ae-8d1567492c4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb90ae7084041aab512ebdaabcfc782'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:40:01.585376 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0922daf9-d400-477f-89ae-8d1567492c4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb90ae7084041aab512ebdaabcfc782'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:40:01.592477 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0922daf9-d400-477f-89ae-8d1567492c4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb90ae7084041aab512ebdaabcfc782'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:40:01.610501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1457/2900] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:40:01 2022] DELETE /identity/v3/users/bcb90ae7084041aab512ebdaabcfc782 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:40:01.614229 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ab9e4-d8d6-4e3d-9f16-94bbf8013d3a None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:01.614760 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ab9e4-d8d6-4e3d-9f16-94bbf8013d3a None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:01.614760 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ab9e4-d8d6-4e3d-9f16-94bbf8013d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:01.634522 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-396ab9e4-d8d6-4e3d-9f16-94bbf8013d3a None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:40:01.635704 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-396ab9e4-d8d6-4e3d-9f16-94bbf8013d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:01.696111 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1444/2901] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:40:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:40:01.705849 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922daf9-d400-477f-89ae-8d1567492c4e admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:40:01.707226 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-0922daf9-d400-477f-89ae-8d1567492c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:01.763340 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922daf9-d400-477f-89ae-8d1567492c4e admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:40:01.766984 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ae449-bffd-4309-93e4-b950c2215657 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:40:01.767589 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ae449-bffd-4309-93e4-b950c2215657 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:01.767869 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ae449-bffd-4309-93e4-b950c2215657 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:01.768140 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ae449-bffd-4309-93e4-b950c2215657 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:01.778649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ae449-bffd-4309-93e4-b950c2215657 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:40:01.780069 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ae449-bffd-4309-93e4-b950c2215657 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:40:01.790029 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1458/2902] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:40:02.148766 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ab9e4-d8d6-4e3d-9f16-94bbf8013d3a None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:40:02.150220 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-396ab9e4-d8d6-4e3d-9f16-94bbf8013d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:02.162455 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ab9e4-d8d6-4e3d-9f16-94bbf8013d3a None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:40:02.167340 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6049c7-c14e-4dc4-bdfa-711519e71bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:40:02.167981 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6049c7-c14e-4dc4-bdfa-711519e71bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:02.168260 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6049c7-c14e-4dc4-bdfa-711519e71bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:02.168538 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6049c7-c14e-4dc4-bdfa-711519e71bab admin admin] PATH_INFO: `/v3/projects/9e7346b5ae604bf39f284fb45d53d617` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:02.169700 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6049c7-c14e-4dc4-bdfa-711519e71bab admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e7346b5ae604bf39f284fb45d53d617)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:40:02.171609 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6049c7-c14e-4dc4-bdfa-711519e71bab admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:40:02.218938 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1a6049c7-c14e-4dc4-bdfa-711519e71bab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e7346b5ae604bf39f284fb45d53d617', 'request_id': 'req-1a6049c7-c14e-4dc4-bdfa-711519e71bab'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:40:02.225749 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1a6049c7-c14e-4dc4-bdfa-711519e71bab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e7346b5ae604bf39f284fb45d53d617', 'request_id': 'req-1a6049c7-c14e-4dc4-bdfa-711519e71bab'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:40:02.233071 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1a6049c7-c14e-4dc4-bdfa-711519e71bab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e7346b5ae604bf39f284fb45d53d617', 'request_id': 'req-1a6049c7-c14e-4dc4-bdfa-711519e71bab'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:40:02.244033 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1a6049c7-c14e-4dc4-bdfa-711519e71bab admin admin] The token cache is being invalidate because project 9e7346b5ae604bf39f284fb45d53d617 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:40:02.244302 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1a6049c7-c14e-4dc4-bdfa-711519e71bab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:40:02.244918 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-1a6049c7-c14e-4dc4-bdfa-711519e71bab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:40:02.246438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1445/2903] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:40:02 2022] DELETE /identity/v3/projects/9e7346b5ae604bf39f284fb45d53d617 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:40:02.251285 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eae585-68d6-432b-864d-2ec8cedadf09 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:02.251572 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eae585-68d6-432b-864d-2ec8cedadf09 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:02.251851 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eae585-68d6-432b-864d-2ec8cedadf09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:02.271187 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-58eae585-68d6-432b-864d-2ec8cedadf09 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:40:02.272603 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-58eae585-68d6-432b-864d-2ec8cedadf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:02.289923 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:40:02.334463 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1459/2904] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:40:02.343624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6874d-f753-4ec2-be40-07f47c47b13c None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:02.343936 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6874d-f753-4ec2-be40-07f47c47b13c None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:02.344210 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6874d-f753-4ec2-be40-07f47c47b13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:02.352229 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:40:02.368084 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-8ba6874d-f753-4ec2-be40-07f47c47b13c None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:40:02.369354 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-8ba6874d-f753-4ec2-be40-07f47c47b13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:02.377154 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1446/2905] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:40:02.381390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eae585-68d6-432b-864d-2ec8cedadf09 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:40:02.391009 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eae585-68d6-432b-864d-2ec8cedadf09 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:40:02.395588 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99c4f2-6be1-4ea6-89cf-eb5366d32ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:40:02.396129 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99c4f2-6be1-4ea6-89cf-eb5366d32ada admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:02.396410 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99c4f2-6be1-4ea6-89cf-eb5366d32ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:02.396683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99c4f2-6be1-4ea6-89cf-eb5366d32ada admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:02.397396 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99c4f2-6be1-4ea6-89cf-eb5366d32ada admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:40:02.397692 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99c4f2-6be1-4ea6-89cf-eb5366d32ada admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:40:02.399269 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99c4f2-6be1-4ea6-89cf-eb5366d32ada admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:40:02.410523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1460/2906] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:40:02.415096 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6874d-f753-4ec2-be40-07f47c47b13c None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:40:02.424290 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6874d-f753-4ec2-be40-07f47c47b13c None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:40:02.429858 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e48be-c9cc-45c2-ada5-74e6b0873506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:40:02.430456 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e48be-c9cc-45c2-ada5-74e6b0873506 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:02.430740 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e48be-c9cc-45c2-ada5-74e6b0873506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:02.431013 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e48be-c9cc-45c2-ada5-74e6b0873506 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:02.431633 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e48be-c9cc-45c2-ada5-74e6b0873506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:40:02.433539 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e48be-c9cc-45c2-ada5-74e6b0873506 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:40:02.459767 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1447/2907] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:40:02 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:40:02.463945 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99c4f2-6be1-4ea6-89cf-eb5366d32ada admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:40:02.473990 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99c4f2-6be1-4ea6-89cf-eb5366d32ada admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:40:02.530434 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0571c-d5ab-4ec4-9b85-5649308b1bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:40:02.531103 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0571c-d5ab-4ec4-9b85-5649308b1bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:02.531302 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0571c-d5ab-4ec4-9b85-5649308b1bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:02.531579 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0571c-d5ab-4ec4-9b85-5649308b1bf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:02.532205 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0571c-d5ab-4ec4-9b85-5649308b1bf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:40:02.534157 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e0571c-d5ab-4ec4-9b85-5649308b1bf0 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:40:02.555649 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1461/2908] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:40:02 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:40:02.559905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e48be-c9cc-45c2-ada5-74e6b0873506 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:40:02.570391 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e48be-c9cc-45c2-ada5-74e6b0873506 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:40:02.574958 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c0ecc-71fc-4e6d-95cd-aed7ba476116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:40:02.575552 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c0ecc-71fc-4e6d-95cd-aed7ba476116 admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:02.575832 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c0ecc-71fc-4e6d-95cd-aed7ba476116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:02.576102 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c0ecc-71fc-4e6d-95cd-aed7ba476116 admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:02.576735 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c0ecc-71fc-4e6d-95cd-aed7ba476116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:40:02.578355 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c0ecc-71fc-4e6d-95cd-aed7ba476116 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:40:02.589093 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1448/2909] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:40:02.593360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0571c-d5ab-4ec4-9b85-5649308b1bf0 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:40:02.604264 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0571c-d5ab-4ec4-9b85-5649308b1bf0 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:40:02.608825 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcca277-d411-426e-95cf-747028fdb5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:40:02.609465 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcca277-d411-426e-95cf-747028fdb5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:02.609752 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcca277-d411-426e-95cf-747028fdb5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:02.610031 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcca277-d411-426e-95cf-747028fdb5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:02.610666 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcca277-d411-426e-95cf-747028fdb5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:40:02.612256 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcca277-d411-426e-95cf-747028fdb5e6 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:40:02.623379 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1462/2910] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:40:02.627797 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c0ecc-71fc-4e6d-95cd-aed7ba476116 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:40:02.637985 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c0ecc-71fc-4e6d-95cd-aed7ba476116 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:40:02.642566 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bfa85-ddd7-40f7-b695-070d3feb1b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:40:02.643341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bfa85-ddd7-40f7-b695-070d3feb1b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:02.643624 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bfa85-ddd7-40f7-b695-070d3feb1b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:02.643894 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bfa85-ddd7-40f7-b695-070d3feb1b69 admin admin] PATH_INFO: `/v3/projects/a24baceb1473444fa611c71a7c8fb5ae/users/5097f22898f943f6a9a843048fa9c8b0/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:02.662617 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bfa85-ddd7-40f7-b695-070d3feb1b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a24baceb1473444fa611c71a7c8fb5ae, user_id=5097f22898f943f6a9a843048fa9c8b0, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:40:02.666190 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bfa85-ddd7-40f7-b695-070d3feb1b69 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:40:02.679918 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1449/2911] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:40:02 2022] PUT /identity/v3/projects/a24baceb1473444fa611c71a7c8fb5ae/users/5097f22898f943f6a9a843048fa9c8b0/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:40:02.685244 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab25407-cae9-46b8-9bec-df81859341de None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:02.685523 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab25407-cae9-46b8-9bec-df81859341de None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:02.685800 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab25407-cae9-46b8-9bec-df81859341de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:02.704692 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-3ab25407-cae9-46b8-9bec-df81859341de None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:40:02.705926 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-3ab25407-cae9-46b8-9bec-df81859341de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:02.763853 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1463/2912] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:40:02.773948 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bfa85-ddd7-40f7-b695-070d3feb1b69 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:40:02.775619 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-670bfa85-ddd7-40f7-b695-070d3feb1b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:02.832153 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bfa85-ddd7-40f7-b695-070d3feb1b69 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:40:02.836364 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad7e24-335d-4039-9db5-41041dd9b1e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:40:02.836960 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad7e24-335d-4039-9db5-41041dd9b1e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:02.837270 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad7e24-335d-4039-9db5-41041dd9b1e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:02.837542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad7e24-335d-4039-9db5-41041dd9b1e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:02.846520 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad7e24-335d-4039-9db5-41041dd9b1e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:40:02.847931 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad7e24-335d-4039-9db5-41041dd9b1e7 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:40:02.857680 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1450/2913] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:40:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:40:03.492248 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80910664-cec1-4424-a470-c6254052fad9 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:03.492535 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80910664-cec1-4424-a470-c6254052fad9 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:03.492838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80910664-cec1-4424-a470-c6254052fad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:03.522907 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-80910664-cec1-4424-a470-c6254052fad9 None None] MFA Rules not processed for user `5097f22898f943f6a9a843048fa9c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:40:03.524080 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-80910664-cec1-4424-a470-c6254052fad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:03.542753 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:40:03.598283 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1464/2914] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:40:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:40:03.610546 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad7e24-335d-4039-9db5-41041dd9b1e7 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:40:03.612209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-42ad7e24-335d-4039-9db5-41041dd9b1e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:03.681414 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad7e24-335d-4039-9db5-41041dd9b1e7 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:40:03.685527 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef362b-8b9d-455d-bbd6-05e9fba9497a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:40:03.686127 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef362b-8b9d-455d-bbd6-05e9fba9497a service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:03.686411 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef362b-8b9d-455d-bbd6-05e9fba9497a service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:03.686688 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef362b-8b9d-455d-bbd6-05e9fba9497a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:03.697195 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef362b-8b9d-455d-bbd6-05e9fba9497a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:40:03.698667 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef362b-8b9d-455d-bbd6-05e9fba9497a service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:40:03.710080 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1451/2915] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:40:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:40:05.113008 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25b5a7-fe78-43f3-b0f4-69116e7383dc None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:05.113594 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25b5a7-fe78-43f3-b0f4-69116e7383dc None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:05.113714 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25b5a7-fe78-43f3-b0f4-69116e7383dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:05.132301 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-fb25b5a7-fe78-43f3-b0f4-69116e7383dc None None] MFA Rules not processed for user `5097f22898f943f6a9a843048fa9c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:40:05.133506 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-fb25b5a7-fe78-43f3-b0f4-69116e7383dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:05.140170 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1465/2916] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:40:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:40:07.742276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef362b-8b9d-455d-bbd6-05e9fba9497a service cinder] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:40:07.743957 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-87ef362b-8b9d-455d-bbd6-05e9fba9497a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:07.810290 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:40:07.845007 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef362b-8b9d-455d-bbd6-05e9fba9497a service cinder] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:40:07.851635 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ff3d5e-d002-4b1e-b0d6-9c3ee0d2a0eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:40:07.852369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff3d5e-d002-4b1e-b0d6-9c3ee0d2a0eb service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:07.852655 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff3d5e-d002-4b1e-b0d6-9c3ee0d2a0eb service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:07.852927 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ff3d5e-d002-4b1e-b0d6-9c3ee0d2a0eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:07.854696 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c4ff3d5e-d002-4b1e-b0d6-9c3ee0d2a0eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:07.869074 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ff3d5e-d002-4b1e-b0d6-9c3ee0d2a0eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:40:07.871089 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ff3d5e-d002-4b1e-b0d6-9c3ee0d2a0eb service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:40:07.883110 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1452/2917] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:40:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 12:40:10.441501 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25b5a7-fe78-43f3-b0f4-69116e7383dc None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:40:10.442922 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-fb25b5a7-fe78-43f3-b0f4-69116e7383dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:10.455365 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25b5a7-fe78-43f3-b0f4-69116e7383dc None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:40:10.458907 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54502476-576f-4bc9-a76c-c5b7b0efc437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:40:10.459529 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54502476-576f-4bc9-a76c-c5b7b0efc437 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:10.459809 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54502476-576f-4bc9-a76c-c5b7b0efc437 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:10.460082 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54502476-576f-4bc9-a76c-c5b7b0efc437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:10.462392 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-54502476-576f-4bc9-a76c-c5b7b0efc437 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:10.524728 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54502476-576f-4bc9-a76c-c5b7b0efc437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:40:10.526204 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54502476-576f-4bc9-a76c-c5b7b0efc437 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:40:10.535764 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1466/2918] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:40:10 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:40:12.067177 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ff3d5e-d002-4b1e-b0d6-9c3ee0d2a0eb service nova] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:40:12.069708 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-c4ff3d5e-d002-4b1e-b0d6-9c3ee0d2a0eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:12.083336 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ff3d5e-d002-4b1e-b0d6-9c3ee0d2a0eb service nova] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:40:12.089919 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8525e-a822-44b7-b86c-b988304c1563 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '982b82e8ac424318a934f9149fb91706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:40:12.090524 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8525e-a822-44b7-b86c-b988304c1563 service nova] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:12.090805 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8525e-a822-44b7-b86c-b988304c1563 service nova] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:12.091078 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8525e-a822-44b7-b86c-b988304c1563 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:12.092770 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-00b8525e-a822-44b7-b86c-b988304c1563 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:12.105342 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8525e-a822-44b7-b86c-b988304c1563 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:40:12.106908 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8525e-a822-44b7-b86c-b988304c1563 service nova] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:40:12.117855 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1453/2919] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:40:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:40:20.531439 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=52 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:40:22.113696 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=20 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:40:22.789277 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da801a-d032-4933-8ca8-68a1459555a7 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:22.789758 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da801a-d032-4933-8ca8-68a1459555a7 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:22.789866 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da801a-d032-4933-8ca8-68a1459555a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:22.810331 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-24da801a-d032-4933-8ca8-68a1459555a7 None None] MFA Rules not processed for user `5097f22898f943f6a9a843048fa9c8b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:40:22.811524 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-24da801a-d032-4933-8ca8-68a1459555a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:22.818237 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1467/2920] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:40:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:40:23.049375 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8525e-a822-44b7-b86c-b988304c1563 service nova] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:40:23.059487 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8525e-a822-44b7-b86c-b988304c1563 service nova] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:40:23.063697 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c193a26-a30d-486e-b23d-5e82a3cfb3fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:40:23.064292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c193a26-a30d-486e-b23d-5e82a3cfb3fd service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:23.064569 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c193a26-a30d-486e-b23d-5e82a3cfb3fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:23.064836 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c193a26-a30d-486e-b23d-5e82a3cfb3fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:23.066640 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2c193a26-a30d-486e-b23d-5e82a3cfb3fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:23.076842 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c193a26-a30d-486e-b23d-5e82a3cfb3fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:40:23.078287 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c193a26-a30d-486e-b23d-5e82a3cfb3fd service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:40:23.086146 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1454/2921] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:40:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:40:32.803567 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:40:33.082543 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=6 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:40:46.005401 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cef347-7442-4c8d-af2b-fecc0d96595f None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:46.005911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cef347-7442-4c8d-af2b-fecc0d96595f None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:46.006020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cef347-7442-4c8d-af2b-fecc0d96595f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:46.054075 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-a1cef347-7442-4c8d-af2b-fecc0d96595f None None] MFA Rules not processed for user `bc69cf267bea41bf890bc7551d84ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:40:46.055258 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a1cef347-7442-4c8d-af2b-fecc0d96595f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:46.120031 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1468/2922] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Sep 28 12:40:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:40:46.171514 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c193a26-a30d-486e-b23d-5e82a3cfb3fd service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:40:46.173002 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-2c193a26-a30d-486e-b23d-5e82a3cfb3fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:40:46.277650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c193a26-a30d-486e-b23d-5e82a3cfb3fd service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:40:46.282590 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399f7a3-bb69-4070-a312-0db1a30a1f63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975a8d7349994520bcbfb5f75e52aac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:40:46.283204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399f7a3-bb69-4070-a312-0db1a30a1f63 service swift] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:40:46.283483 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399f7a3-bb69-4070-a312-0db1a30a1f63 service swift] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:40:46.283752 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399f7a3-bb69-4070-a312-0db1a30a1f63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:40:46.292272 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399f7a3-bb69-4070-a312-0db1a30a1f63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:40:46.293883 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399f7a3-bb69-4070-a312-0db1a30a1f63 service swift] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:40:46.301705 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1455/2923] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 12:40:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 12:40:56.117005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=40 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:40:56.298523 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=47 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:41:01.624490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd0a3e-ac01-4a0d-aa40-5f6e6b069aab None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:01.625005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd0a3e-ac01-4a0d-aa40-5f6e6b069aab None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:01.625112 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd0a3e-ac01-4a0d-aa40-5f6e6b069aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:01.652593 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-87fd0a3e-ac01-4a0d-aa40-5f6e6b069aab None None] MFA Rules not processed for user `bc69cf267bea41bf890bc7551d84ad6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:41:01.653807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-87fd0a3e-ac01-4a0d-aa40-5f6e6b069aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:41:01.679522 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1469/2924] 10.222.0.60 () {58 vars in 967 bytes} [Wed Sep 28 12:41:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 12:41:01.692343 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399f7a3-bb69-4070-a312-0db1a30a1f63 service swift] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:41:01.719326 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399f7a3-bb69-4070-a312-0db1a30a1f63 service swift] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:41:01.724310 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8964b-84ea-467a-b9e4-0fece09bfddb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '975a8d7349994520bcbfb5f75e52aac7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:41:01.724901 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8964b-84ea-467a-b9e4-0fece09bfddb service swift] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:01.725210 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8964b-84ea-467a-b9e4-0fece09bfddb service swift] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:01.725488 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8964b-84ea-467a-b9e4-0fece09bfddb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:01.733926 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8964b-84ea-467a-b9e4-0fece09bfddb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:41:01.735485 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8964b-84ea-467a-b9e4-0fece09bfddb service swift] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:41:01.743437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1456/2925] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Sep 28 12:41:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 12:41:03.949490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd0a3e-ac01-4a0d-aa40-5f6e6b069aab None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:41:03.990843 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd0a3e-ac01-4a0d-aa40-5f6e6b069aab None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:41:03.996920 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ee26a-0b0f-47ae-9339-3bef1274ddfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:41:03.997585 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ee26a-0b0f-47ae-9339-3bef1274ddfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:03.997868 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ee26a-0b0f-47ae-9339-3bef1274ddfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:03.998140 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ee26a-0b0f-47ae-9339-3bef1274ddfa admin admin] PATH_INFO: `/v3/users/5097f22898f943f6a9a843048fa9c8b0` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:03.999270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ee26a-0b0f-47ae-9339-3bef1274ddfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=5097f22898f943f6a9a843048fa9c8b0)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:41:04.002664 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ee26a-0b0f-47ae-9339-3bef1274ddfa admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:41:04.053619 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0b6ee26a-0b0f-47ae-9339-3bef1274ddfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5097f22898f943f6a9a843048fa9c8b0'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:04.053956 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0b6ee26a-0b0f-47ae-9339-3bef1274ddfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5097f22898f943f6a9a843048fa9c8b0'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:04.063084 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0b6ee26a-0b0f-47ae-9339-3bef1274ddfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5097f22898f943f6a9a843048fa9c8b0'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:04.063730 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0b6ee26a-0b0f-47ae-9339-3bef1274ddfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5097f22898f943f6a9a843048fa9c8b0'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:04.070798 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-0b6ee26a-0b0f-47ae-9339-3bef1274ddfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5097f22898f943f6a9a843048fa9c8b0'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:04.090620 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1470/2926] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:41:03 2022] DELETE /identity/v3/users/5097f22898f943f6a9a843048fa9c8b0 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:41:04.283564 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8964b-84ea-467a-b9e4-0fece09bfddb service swift] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:41:04.299999 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8964b-84ea-467a-b9e4-0fece09bfddb service swift] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:41:04.364464 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b7d3c-b89f-4d4e-abf6-fe84444626b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:41:04.365109 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b7d3c-b89f-4d4e-abf6-fe84444626b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:04.365421 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b7d3c-b89f-4d4e-abf6-fe84444626b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:04.365695 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b7d3c-b89f-4d4e-abf6-fe84444626b5 admin admin] PATH_INFO: `/v3/projects/a24baceb1473444fa611c71a7c8fb5ae` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:04.366901 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b7d3c-b89f-4d4e-abf6-fe84444626b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a24baceb1473444fa611c71a7c8fb5ae)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:41:04.369678 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b7d3c-b89f-4d4e-abf6-fe84444626b5 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:41:04.414346 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-b47b7d3c-b89f-4d4e-abf6-fe84444626b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a24baceb1473444fa611c71a7c8fb5ae', 'request_id': 'req-b47b7d3c-b89f-4d4e-abf6-fe84444626b5'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:04.420867 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-b47b7d3c-b89f-4d4e-abf6-fe84444626b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a24baceb1473444fa611c71a7c8fb5ae', 'request_id': 'req-b47b7d3c-b89f-4d4e-abf6-fe84444626b5'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:04.428418 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-b47b7d3c-b89f-4d4e-abf6-fe84444626b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a24baceb1473444fa611c71a7c8fb5ae', 'request_id': 'req-b47b7d3c-b89f-4d4e-abf6-fe84444626b5'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:04.451509 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-b47b7d3c-b89f-4d4e-abf6-fe84444626b5 admin admin] The token cache is being invalidate because project a24baceb1473444fa611c71a7c8fb5ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:41:04.451794 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-b47b7d3c-b89f-4d4e-abf6-fe84444626b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:04.452417 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-b47b7d3c-b89f-4d4e-abf6-fe84444626b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:04.453921 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1457/2927] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:41:04 2022] DELETE /identity/v3/projects/a24baceb1473444fa611c71a7c8fb5ae => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:41:04.458841 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923e1d3-6032-4d30-bc54-50e763a91983 None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:04.459127 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923e1d3-6032-4d30-bc54-50e763a91983 None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:04.459400 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923e1d3-6032-4d30-bc54-50e763a91983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:04.479989 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-2923e1d3-6032-4d30-bc54-50e763a91983 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:41:04.481214 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2923e1d3-6032-4d30-bc54-50e763a91983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:41:04.541512 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1471/2928] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:41:04.548891 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747b972-45d2-4f00-bfe4-b6059f2b78fd None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:04.549550 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747b972-45d2-4f00-bfe4-b6059f2b78fd None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:04.550130 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747b972-45d2-4f00-bfe4-b6059f2b78fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:04.570839 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-4747b972-45d2-4f00-bfe4-b6059f2b78fd None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:41:04.572125 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-4747b972-45d2-4f00-bfe4-b6059f2b78fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:41:04.580369 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1458/2929] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:41:04.584619 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923e1d3-6032-4d30-bc54-50e763a91983 None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:41:04.595474 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923e1d3-6032-4d30-bc54-50e763a91983 None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:41:04.601366 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b8e12-7755-4560-9c1e-42df7ef506e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:41:04.602235 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b8e12-7755-4560-9c1e-42df7ef506e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:04.603119 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b8e12-7755-4560-9c1e-42df7ef506e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:04.603690 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b8e12-7755-4560-9c1e-42df7ef506e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:04.604460 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b8e12-7755-4560-9c1e-42df7ef506e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81384) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 12:41:04.604911 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b8e12-7755-4560-9c1e-42df7ef506e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:41:04.606557 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b8e12-7755-4560-9c1e-42df7ef506e9 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:41:04.622938 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1472/2930] 10.222.0.60 () {58 vars in 1153 bytes} [Wed Sep 28 12:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 12:41:04.628124 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747b972-45d2-4f00-bfe4-b6059f2b78fd None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:41:04.637977 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747b972-45d2-4f00-bfe4-b6059f2b78fd None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:41:04.643063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4048248-f10e-4f9c-b173-76805312a201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:41:04.643664 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4048248-f10e-4f9c-b173-76805312a201 admin admin] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:04.644063 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4048248-f10e-4f9c-b173-76805312a201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:04.644338 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4048248-f10e-4f9c-b173-76805312a201 admin admin] PATH_INFO: `/v3/projects` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:04.644976 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4048248-f10e-4f9c-b173-76805312a201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:41:04.647012 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4048248-f10e-4f9c-b173-76805312a201 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:41:04.676683 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1459/2931] 10.222.0.60 () {60 vars in 1153 bytes} [Wed Sep 28 12:41:04 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 12:41:04.681447 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b8e12-7755-4560-9c1e-42df7ef506e9 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:41:04.691715 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b8e12-7755-4560-9c1e-42df7ef506e9 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:41:04.696558 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:41:04.751878 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d159cb-cd0c-4685-a677-fa3da3873603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:41:04.752485 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d159cb-cd0c-4685-a677-fa3da3873603 admin admin] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:04.752762 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d159cb-cd0c-4685-a677-fa3da3873603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:04.753032 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d159cb-cd0c-4685-a677-fa3da3873603 admin admin] PATH_INFO: `/v3/users` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:04.753683 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d159cb-cd0c-4685-a677-fa3da3873603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:41:04.755570 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d159cb-cd0c-4685-a677-fa3da3873603 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:41:04.777707 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1473/2932] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Sep 28 12:41:04 2022] POST /identity/v3/users => generated 282 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 12:41:04.781974 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4048248-f10e-4f9c-b173-76805312a201 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:41:04.793660 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4048248-f10e-4f9c-b173-76805312a201 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:41:04.798320 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156e2ba-ecc7-4062-8051-3b35e2fa8acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:41:04.798922 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156e2ba-ecc7-4062-8051-3b35e2fa8acc admin admin] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:04.799204 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156e2ba-ecc7-4062-8051-3b35e2fa8acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:04.799473 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156e2ba-ecc7-4062-8051-3b35e2fa8acc admin admin] PATH_INFO: `/v3/roles` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:04.800108 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156e2ba-ecc7-4062-8051-3b35e2fa8acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:41:04.801720 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156e2ba-ecc7-4062-8051-3b35e2fa8acc admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:41:04.811645 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1460/2933] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:41:04.815646 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d159cb-cd0c-4685-a677-fa3da3873603 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:41:04.825526 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d159cb-cd0c-4685-a677-fa3da3873603 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:41:04.829999 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa841d4-b2c4-4079-9f96-39bb27ac6480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:41:04.830587 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa841d4-b2c4-4079-9f96-39bb27ac6480 admin admin] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:04.830863 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa841d4-b2c4-4079-9f96-39bb27ac6480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:04.831132 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa841d4-b2c4-4079-9f96-39bb27ac6480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:04.831754 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa841d4-b2c4-4079-9f96-39bb27ac6480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:41:04.833360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa841d4-b2c4-4079-9f96-39bb27ac6480 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:41:04.844854 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1474/2934] 10.222.0.60 () {58 vars in 1122 bytes} [Wed Sep 28 12:41:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 12:41:04.848801 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156e2ba-ecc7-4062-8051-3b35e2fa8acc admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:41:04.859542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156e2ba-ecc7-4062-8051-3b35e2fa8acc admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:41:04.864209 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4df75-209e-4d68-9e30-5f57d06a4f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:41:04.864878 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4df75-209e-4d68-9e30-5f57d06a4f9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:04.865180 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4df75-209e-4d68-9e30-5f57d06a4f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:04.865457 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4df75-209e-4d68-9e30-5f57d06a4f9c admin admin] PATH_INFO: `/v3/projects/6e24bf3125f5431987c8eb481f99ac36/users/e72ddc7f496c43e6b6b208c2f9fd4681/roles/2201e005d5ef449896a0e68a940678ec` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:04.868450 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:41:04.881666 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4df75-209e-4d68-9e30-5f57d06a4f9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e24bf3125f5431987c8eb481f99ac36, user_id=e72ddc7f496c43e6b6b208c2f9fd4681, role_id=2201e005d5ef449896a0e68a940678ec)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:41:04.884438 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4df75-209e-4d68-9e30-5f57d06a4f9c admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:41:04.896769 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1461/2935] 10.222.0.60 () {60 vars in 1483 bytes} [Wed Sep 28 12:41:04 2022] PUT /identity/v3/projects/6e24bf3125f5431987c8eb481f99ac36/users/e72ddc7f496c43e6b6b208c2f9fd4681/roles/2201e005d5ef449896a0e68a940678ec => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:41:04.908100 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61349447-a0fe-4d74-9fa2-95a63453336e None None] REQUEST_METHOD: `POST` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:04.908390 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61349447-a0fe-4d74-9fa2-95a63453336e None None] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:04.908684 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61349447-a0fe-4d74-9fa2-95a63453336e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:04.936020 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.auth.core [None req-61349447-a0fe-4d74-9fa2-95a63453336e None None] MFA Rules not processed for user `e72ddc7f496c43e6b6b208c2f9fd4681`. Rule list: `[]` (Enabled: `True`). {{(pid=81384) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:41:04.937244 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-61349447-a0fe-4d74-9fa2-95a63453336e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:41:05.002737 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1475/2936] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:41:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:41:05.014410 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4df75-209e-4d68-9e30-5f57d06a4f9c admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:41:05.016131 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-fae4df75-209e-4d68-9e30-5f57d06a4f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:41:05.074276 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4df75-209e-4d68-9e30-5f57d06a4f9c admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:41:05.077918 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63b6d92-71bb-49b8-9490-74d99c81e978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2354a4fd72cc42c58d6830eecb1e6253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:41:05.078453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b6d92-71bb-49b8-9490-74d99c81e978 service cinder] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:05.078731 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b6d92-71bb-49b8-9490-74d99c81e978 service cinder] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:05.079003 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b6d92-71bb-49b8-9490-74d99c81e978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:05.087019 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b6d92-71bb-49b8-9490-74d99c81e978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:41:05.088432 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63b6d92-71bb-49b8-9490-74d99c81e978 service cinder] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:41:05.097316 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1462/2937] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:41:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:41:14.999755 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=62,keystone:INSERT=3 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:41:15.093348 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:41:48.600546 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61349447-a0fe-4d74-9fa2-95a63453336e None None] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:41:48.660961 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61349447-a0fe-4d74-9fa2-95a63453336e None None] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:41:48.665438 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f880dee-d7e2-4c7e-9b6a-46c42db7cfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:41:48.666063 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f880dee-d7e2-4c7e-9b6a-46c42db7cfad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:48.666343 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f880dee-d7e2-4c7e-9b6a-46c42db7cfad admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:48.666617 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f880dee-d7e2-4c7e-9b6a-46c42db7cfad admin admin] PATH_INFO: `/v3/users/e72ddc7f496c43e6b6b208c2f9fd4681` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:48.667723 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f880dee-d7e2-4c7e-9b6a-46c42db7cfad admin admin] RBAC: Authorizing `identity:delete_user(user_id=e72ddc7f496c43e6b6b208c2f9fd4681)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:41:48.669986 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f880dee-d7e2-4c7e-9b6a-46c42db7cfad admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:41:48.715256 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2f880dee-d7e2-4c7e-9b6a-46c42db7cfad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e72ddc7f496c43e6b6b208c2f9fd4681'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:48.715896 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2f880dee-d7e2-4c7e-9b6a-46c42db7cfad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e72ddc7f496c43e6b6b208c2f9fd4681'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:48.723490 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2f880dee-d7e2-4c7e-9b6a-46c42db7cfad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e72ddc7f496c43e6b6b208c2f9fd4681'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:48.724106 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2f880dee-d7e2-4c7e-9b6a-46c42db7cfad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e72ddc7f496c43e6b6b208c2f9fd4681'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:48.732833 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-2f880dee-d7e2-4c7e-9b6a-46c42db7cfad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e72ddc7f496c43e6b6b208c2f9fd4681'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:48.751595 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1476/2938] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:41:48 2022] DELETE /identity/v3/users/e72ddc7f496c43e6b6b208c2f9fd4681 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:41:48.756098 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296a365-b108-41e9-b225-550b64f88422 None None] REQUEST_METHOD: `POST` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:48.756608 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296a365-b108-41e9-b225-550b64f88422 None None] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:48.756723 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296a365-b108-41e9-b225-550b64f88422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:48.787905 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.auth.core [None req-1296a365-b108-41e9-b225-550b64f88422 None None] MFA Rules not processed for user `1290a1aa40a149388a72943a7b2dc68e`. Rule list: `[]` (Enabled: `True`). {{(pid=81385) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 12:41:48.789225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1296a365-b108-41e9-b225-550b64f88422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:41:48.847876 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1463/2939] 10.222.0.60 () {58 vars in 958 bytes} [Wed Sep 28 12:41:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 12:41:48.857979 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f880dee-d7e2-4c7e-9b6a-46c42db7cfad admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:41:48.859388 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-2f880dee-d7e2-4c7e-9b6a-46c42db7cfad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:41:48.937617 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f880dee-d7e2-4c7e-9b6a-46c42db7cfad admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:41:48.941507 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d54fa-dc8b-4c7a-8d88-b1e839e1ba94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:41:48.942109 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d54fa-dc8b-4c7a-8d88-b1e839e1ba94 service neutron] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:48.942386 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d54fa-dc8b-4c7a-8d88-b1e839e1ba94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:48.942663 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d54fa-dc8b-4c7a-8d88-b1e839e1ba94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:48.952384 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d54fa-dc8b-4c7a-8d88-b1e839e1ba94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:41:48.953816 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d54fa-dc8b-4c7a-8d88-b1e839e1ba94 service neutron] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:41:48.963554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1477/2940] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:41:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:41:49.228557 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296a365-b108-41e9-b225-550b64f88422 None None] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:41:49.230023 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-1296a365-b108-41e9-b225-550b64f88422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:41:49.243192 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296a365-b108-41e9-b225-550b64f88422 None None] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:41:49.247919 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3089c-e9db-4fea-a19a-86893fa134ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:41:49.248551 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3089c-e9db-4fea-a19a-86893fa134ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:41:49.248833 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3089c-e9db-4fea-a19a-86893fa134ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:41:49.249114 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3089c-e9db-4fea-a19a-86893fa134ff admin admin] PATH_INFO: `/v3/projects/6e24bf3125f5431987c8eb481f99ac36` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:41:49.250313 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3089c-e9db-4fea-a19a-86893fa134ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e24bf3125f5431987c8eb481f99ac36)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:41:49.252279 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3089c-e9db-4fea-a19a-86893fa134ff admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:41:49.300383 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-78d3089c-e9db-4fea-a19a-86893fa134ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e24bf3125f5431987c8eb481f99ac36', 'request_id': 'req-78d3089c-e9db-4fea-a19a-86893fa134ff'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:49.306477 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-78d3089c-e9db-4fea-a19a-86893fa134ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e24bf3125f5431987c8eb481f99ac36', 'request_id': 'req-78d3089c-e9db-4fea-a19a-86893fa134ff'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:49.313159 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-78d3089c-e9db-4fea-a19a-86893fa134ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e24bf3125f5431987c8eb481f99ac36', 'request_id': 'req-78d3089c-e9db-4fea-a19a-86893fa134ff'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:49.323453 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-78d3089c-e9db-4fea-a19a-86893fa134ff admin admin] The token cache is being invalidate because project 6e24bf3125f5431987c8eb481f99ac36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81385) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:41:49.323742 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-78d3089c-e9db-4fea-a19a-86893fa134ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:49.324374 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-78d3089c-e9db-4fea-a19a-86893fa134ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:41:49.326426 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1464/2941] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:41:49 2022] DELETE /identity/v3/projects/6e24bf3125f5431987c8eb481f99ac36 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:41:58.958838 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:41:59.320414 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:42:05.122618 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d54fa-dc8b-4c7a-8d88-b1e839e1ba94 service neutron] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:42:05.123991 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-1e6d54fa-dc8b-4c7a-8d88-b1e839e1ba94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:42:05.212985 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d54fa-dc8b-4c7a-8d88-b1e839e1ba94 service neutron] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:42:05.271873 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053db94-b161-406e-b453-f0b1f0f49823 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:42:05.272470 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053db94-b161-406e-b453-f0b1f0f49823 service placement] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:42:05.272751 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053db94-b161-406e-b453-f0b1f0f49823 service placement] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:42:05.273029 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053db94-b161-406e-b453-f0b1f0f49823 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:42:05.274807 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5053db94-b161-406e-b453-f0b1f0f49823 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:42:05.286757 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053db94-b161-406e-b453-f0b1f0f49823 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:42:05.288360 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053db94-b161-406e-b453-f0b1f0f49823 service placement] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:42:05.298405 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1478/2942] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:42:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:42:15.293892 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=67 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:42:38.778261 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3089c-e9db-4fea-a19a-86893fa134ff admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:42:38.779799 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-78d3089c-e9db-4fea-a19a-86893fa134ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:42:38.837183 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3089c-e9db-4fea-a19a-86893fa134ff admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:42:38.840894 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df040c5-5494-42cb-a8f0-a48445f3d1d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba75586d3a7d4278ad6c0c31a1bed3df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:42:38.841530 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df040c5-5494-42cb-a8f0-a48445f3d1d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:42:38.841811 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df040c5-5494-42cb-a8f0-a48445f3d1d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:42:38.842081 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df040c5-5494-42cb-a8f0-a48445f3d1d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:42:38.843979 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0df040c5-5494-42cb-a8f0-a48445f3d1d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:42:38.904697 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df040c5-5494-42cb-a8f0-a48445f3d1d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:42:38.906292 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df040c5-5494-42cb-a8f0-a48445f3d1d2 service neutron] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:42:38.915657 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1465/2943] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:42:45.082019 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053db94-b161-406e-b453-f0b1f0f49823 service placement] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:42:45.084208 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-5053db94-b161-406e-b453-f0b1f0f49823 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:42:45.099062 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053db94-b161-406e-b453-f0b1f0f49823 service placement] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:42:45.105397 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc5096-46fc-42b1-bcd2-b32a56649d1e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:42:45.106270 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc5096-46fc-42b1-bcd2-b32a56649d1e service placement] REQUEST_METHOD: `GET` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:42:45.106549 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc5096-46fc-42b1-bcd2-b32a56649d1e service placement] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:42:45.106818 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc5096-46fc-42b1-bcd2-b32a56649d1e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:42:45.109306 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-a8bc5096-46fc-42b1-bcd2-b32a56649d1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:42:45.121543 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc5096-46fc-42b1-bcd2-b32a56649d1e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:42:45.123278 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc5096-46fc-42b1-bcd2-b32a56649d1e service placement] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:42:45.141091 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df040c5-5494-42cb-a8f0-a48445f3d1d2 service neutron] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:42:45.142013 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-0df040c5-5494-42cb-a8f0-a48445f3d1d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:42:45.156571 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1479/2944] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:42:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:42:45.159879 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df040c5-5494-42cb-a8f0-a48445f3d1d2 service neutron] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:42:45.167225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f20fe-926f-493d-89f6-b029355adaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:42:45.167984 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f20fe-926f-493d-89f6-b029355adaec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:42:45.168392 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f20fe-926f-493d-89f6-b029355adaec admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:42:45.168754 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f20fe-926f-493d-89f6-b029355adaec admin admin] PATH_INFO: `/v3/users/5b723fcdba1d4b26bd0cf604d59f8d8c` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:42:45.170128 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f20fe-926f-493d-89f6-b029355adaec admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b723fcdba1d4b26bd0cf604d59f8d8c)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:42:45.172984 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6f20fe-926f-493d-89f6-b029355adaec admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:42:45.226705 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6f6f20fe-926f-493d-89f6-b029355adaec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b723fcdba1d4b26bd0cf604d59f8d8c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:45.227347 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6f6f20fe-926f-493d-89f6-b029355adaec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b723fcdba1d4b26bd0cf604d59f8d8c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:45.239779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6f6f20fe-926f-493d-89f6-b029355adaec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b723fcdba1d4b26bd0cf604d59f8d8c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:45.247029 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6f6f20fe-926f-493d-89f6-b029355adaec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b723fcdba1d4b26bd0cf604d59f8d8c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:45.247689 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-6f6f20fe-926f-493d-89f6-b029355adaec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b723fcdba1d4b26bd0cf604d59f8d8c'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:45.260341 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1466/2945] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:42:45 2022] DELETE /identity/v3/users/5b723fcdba1d4b26bd0cf604d59f8d8c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:42:45.432421 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc5096-46fc-42b1-bcd2-b32a56649d1e service placement] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:42:45.442972 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc5096-46fc-42b1-bcd2-b32a56649d1e service placement] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:42:45.499524 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c54845-8f4b-4e07-909f-6ad25a5eed71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:42:45.500152 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c54845-8f4b-4e07-909f-6ad25a5eed71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:42:45.500429 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c54845-8f4b-4e07-909f-6ad25a5eed71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:42:45.500702 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c54845-8f4b-4e07-909f-6ad25a5eed71 admin admin] PATH_INFO: `/v3/projects/d73c25391e8044c2ac3678317021732d` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:42:45.501814 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c54845-8f4b-4e07-909f-6ad25a5eed71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d73c25391e8044c2ac3678317021732d)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:42:45.503698 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c54845-8f4b-4e07-909f-6ad25a5eed71 admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:42:45.549524 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-23c54845-8f4b-4e07-909f-6ad25a5eed71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd73c25391e8044c2ac3678317021732d', 'request_id': 'req-23c54845-8f4b-4e07-909f-6ad25a5eed71'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:45.555592 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-23c54845-8f4b-4e07-909f-6ad25a5eed71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd73c25391e8044c2ac3678317021732d', 'request_id': 'req-23c54845-8f4b-4e07-909f-6ad25a5eed71'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:45.563108 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-23c54845-8f4b-4e07-909f-6ad25a5eed71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd73c25391e8044c2ac3678317021732d', 'request_id': 'req-23c54845-8f4b-4e07-909f-6ad25a5eed71'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:45.573086 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-23c54845-8f4b-4e07-909f-6ad25a5eed71 admin admin] The token cache is being invalidate because project d73c25391e8044c2ac3678317021732d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:42:45.573418 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-23c54845-8f4b-4e07-909f-6ad25a5eed71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:45.574025 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-23c54845-8f4b-4e07-909f-6ad25a5eed71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:45.575486 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1480/2946] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:42:45 2022] DELETE /identity/v3/projects/d73c25391e8044c2ac3678317021732d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:42:45.580008 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f20fe-926f-493d-89f6-b029355adaec admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:42:45.581437 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-6f6f20fe-926f-493d-89f6-b029355adaec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:42:45.595225 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f20fe-926f-493d-89f6-b029355adaec admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:42:45.613562 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:42:45.658542 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17873c63-2d4d-4a6b-abd1-91887e681946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:42:45.659095 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17873c63-2d4d-4a6b-abd1-91887e681946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:42:45.659376 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17873c63-2d4d-4a6b-abd1-91887e681946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:42:45.659650 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17873c63-2d4d-4a6b-abd1-91887e681946 admin admin] PATH_INFO: `/v3/users/e16f479e1cbf46068e05d1c60de13152` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:42:45.660779 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17873c63-2d4d-4a6b-abd1-91887e681946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e16f479e1cbf46068e05d1c60de13152)` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:42:45.662719 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17873c63-2d4d-4a6b-abd1-91887e681946 admin admin] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:42:45.708811 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-17873c63-2d4d-4a6b-abd1-91887e681946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e16f479e1cbf46068e05d1c60de13152'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:45.709458 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-17873c63-2d4d-4a6b-abd1-91887e681946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e16f479e1cbf46068e05d1c60de13152'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:45.721583 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-17873c63-2d4d-4a6b-abd1-91887e681946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e16f479e1cbf46068e05d1c60de13152'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:45.728640 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-17873c63-2d4d-4a6b-abd1-91887e681946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e16f479e1cbf46068e05d1c60de13152'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:45.729293 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.notifications [None req-17873c63-2d4d-4a6b-abd1-91887e681946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e16f479e1cbf46068e05d1c60de13152'} {{(pid=81385) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:45.741782 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1467/2947] 10.222.0.60 () {58 vars in 1224 bytes} [Wed Sep 28 12:42:45 2022] DELETE /identity/v3/users/e16f479e1cbf46068e05d1c60de13152 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:42:45.913853 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c54845-8f4b-4e07-909f-6ad25a5eed71 admin admin] Authenticating user token {{(pid=81384) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:42:45.915358 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.fernet_utils [None req-23c54845-8f4b-4e07-909f-6ad25a5eed71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81384) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:42:45.929333 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c54845-8f4b-4e07-909f-6ad25a5eed71 admin admin] Validating token access rules against request {{(pid=81384) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:42:45.985822 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca76605-8f83-4339-ba8c-a36e3b1de39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1290a1aa40a149388a72943a7b2dc68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2fc00be567cf4966a588e131d48e5a69', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81384) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:42:45.986453 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca76605-8f83-4339-ba8c-a36e3b1de39b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:42:45.986733 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca76605-8f83-4339-ba8c-a36e3b1de39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:42:45.987005 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca76605-8f83-4339-ba8c-a36e3b1de39b admin admin] PATH_INFO: `/v3/projects/f621e8defb6c4a7796f19073bfcc35cb` {{(pid=81384) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:42:45.988106 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca76605-8f83-4339-ba8c-a36e3b1de39b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f621e8defb6c4a7796f19073bfcc35cb)` {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:42:45.990023 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca76605-8f83-4339-ba8c-a36e3b1de39b admin admin] RBAC: Authorization granted {{(pid=81384) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:42:46.011535 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:42:46.037784 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-7ca76605-8f83-4339-ba8c-a36e3b1de39b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f621e8defb6c4a7796f19073bfcc35cb', 'request_id': 'req-7ca76605-8f83-4339-ba8c-a36e3b1de39b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:46.043788 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-7ca76605-8f83-4339-ba8c-a36e3b1de39b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f621e8defb6c4a7796f19073bfcc35cb', 'request_id': 'req-7ca76605-8f83-4339-ba8c-a36e3b1de39b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:46.051274 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-7ca76605-8f83-4339-ba8c-a36e3b1de39b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f621e8defb6c4a7796f19073bfcc35cb', 'request_id': 'req-7ca76605-8f83-4339-ba8c-a36e3b1de39b'} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:46.062133 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-7ca76605-8f83-4339-ba8c-a36e3b1de39b admin admin] The token cache is being invalidate because project f621e8defb6c4a7796f19073bfcc35cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81384) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 12:42:46.062437 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-7ca76605-8f83-4339-ba8c-a36e3b1de39b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:46.063060 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG keystone.notifications [None req-7ca76605-8f83-4339-ba8c-a36e3b1de39b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81384) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 12:42:46.064554 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: [pid: 81384|app: 0|req: 1481/2948] 10.222.0.60 () {58 vars in 1233 bytes} [Wed Sep 28 12:42:45 2022] DELETE /identity/v3/projects/f621e8defb6c4a7796f19073bfcc35cb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 12:42:55.738408 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:42:56.058679 dv-cinder-844563-9-iscsi devstack@keystone.service[81384]: DEBUG dbcounter [-] [81384] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81384) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 12:47:10.070969 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17873c63-2d4d-4a6b-abd1-91887e681946 admin admin] Authenticating user token {{(pid=81385) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 12:47:10.072522 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-17873c63-2d4d-4a6b-abd1-91887e681946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:47:10.142492 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17873c63-2d4d-4a6b-abd1-91887e681946 admin admin] Validating token access rules against request {{(pid=81385) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 12:47:10.205306 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cecf0d-37c3-4de1-86de-6d0abf30272f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a2322b1a76d40e49545f5677e247489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6335d0db1f4844c39c255909189e0ce9', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81385) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 12:47:10.205918 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cecf0d-37c3-4de1-86de-6d0abf30272f service placement] REQUEST_METHOD: `GET` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 12:47:10.206198 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cecf0d-37c3-4de1-86de-6d0abf30272f service placement] SCRIPT_NAME: `/identity` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 12:47:10.206471 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cecf0d-37c3-4de1-86de-6d0abf30272f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81385) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 12:47:10.208193 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.fernet_utils [None req-30cecf0d-37c3-4de1-86de-6d0abf30272f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81385) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 12:47:10.219726 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cecf0d-37c3-4de1-86de-6d0abf30272f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 12:47:10.221360 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cecf0d-37c3-4de1-86de-6d0abf30272f service placement] RBAC: Authorization granted {{(pid=81385) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 12:47:10.231303 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: [pid: 81385|app: 0|req: 1468/2949] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Sep 28 12:47:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 12:47:20.226993 dv-cinder-844563-9-iscsi devstack@keystone.service[81385]: DEBUG dbcounter [-] [81385] Writing DB stats keystone:SELECT=58 {{(pid=81385) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}